Google Finance.

This is an new vis tool for showing stock performance correlated with news events: Google Finance (I pre-loaded my own company's data here). You can adjust the window you're seeing by size and location, and see the related stories associated with peaks and dips. It's kind of fun, although I wish they'd laid the page out a little nicer.

Greyfriar's Kirkyard, Edinburgh.

Deacon Brodie's on the Royal Mile, Edinburgh

Why Don't We Choose What Makes Us Happy?

People are flawed. But it's rarely so well summarized as it is in the paper "Decision and Experience: Why Don't We Choose What Makes Us Happy?" (Hsee and Hastie 2006). The authors recap the many experiments in the literature of behavioral-decision theory that show that people don't choose outcomes that maximize their happiness, due to a number of failures in the human psychological makeup.

The vast popular literature on self-improvement is based on the belief that we aren’t getting everything we could out of life, and is replete with recipes to increase happiness. Recent findings from behavioral-decision research provide evidence that people are not always able to choose what yields the greatest happiness or best experience. People fail to choose optimally, either because they fail to predict accurately which option in the available choice set will generate the best experience or because they fail to base their choice on their prediction, or both.

They identify these biasing problems:

• Impact bias: We misjudge how severe an impact will be and hence choose the wrong option.
• Projection bias: We project from our current emotional or physical state how we will feel in the future. Hungry shopper buy more food than needed.
• Distinction bias: Joint-evaluation versus single-evaluation models screw us up. I pick from a selection of plasma TVs, but weigh attributes incorrectly because in my home I only experience one of them and the factors I weighed weren't relevant.
• Memory bias: We misremember peak events or significant events, and it influences future choice. (Subjects were lightly tortured with cold water for this one.) The bias disappears with gentle questioning though, good news for therapists.
• Belief bias: Lay theories about what will make us happy -- or, poor self-analysis. (People probably differ in this area.) This section really highlights how pathetic we are, though: "Another common belief is that more choice options are always better. In reality, having more options can lead to worse experiences [38–40]. For example, if employees are given a free trip to Paris, they are happy; if they are given a free trip to Hawaii, they are happy. But if they are given a choice between the two trips, they will be less happy, no matter which option they choose. Having the choice highlights the relative deficiencies in each option. People who choose Paris complain that 'Paris does not have the ocean', whereas people who choose Hawaii complain that 'Hawaii does not have great museums'."

Failures to follow decisions, another source for the failure of rational paths to happiness, are explained by:

• Impulsivity: We choose the short term immediate over the long-term outcome.
• Rule-based decisions: Related to aphorisms and cultural beliefs about "what's right," this is behavior based on rules like "don't waste" rather than rational predictions.
• Lay rationalism: Related to rules, this category represents the attempt to apply correct reasoning but getting it badly wrong. The 3 types covered here are "lay economism," "lay scientism," and "lay functionalism." "Another manifestation is ‘lay scientism’, a tendency to base choices on objective, 'hard' attributes rather than subjective, 'soft' attributes. For example, when choosing between two equally expensive audio systems, one with a higher wattage rating (a hard attribute) and the other with a richer sound (a soft attribute), most people chose the high-wattage model, even though when asked to predict their enjoyment, they favored the richer-sounding model. A third manifestation of lay rationalism is ‘lay functionalism’, a tendency to focus on the primary goal(s) of the decision and overlook other aspects that are important to overall experience."
• Medium maximization: People confuse the medium for happiness with the actual results, the most famous example being money. People work harder to get more money but more money itself doesn't increase happiness.

The implications for this type of research are politically worrying, of course (we assume in democratic and capitalist societies that people are capable of choosing what is best for them and should be allowed to do so). For software design and other economic problems, the implications are equally sad; if people can't be relied on to choose the products that are "best" by rational means, then the well-intentioned decisions of designers are that much less important and less related to final market success. Product usability itself is a "soft" factor subject to being sacrificed as unimportant, thanks to lay rationalism and the impact and distinction biases.

Glasgow, "The Magic Tea Rooms" (I assume all these juxtaposed colors are no accident--?)

Glasgow Railings, West End.

architecture and hygiene

A friend at work pointed out this amusing site, found after he googled "architecture, hygene" late at night (Google fixed his spelling): architecture and hygiene.

The main gist is prefab houses, but Kalkin's houses are mixed in with a bunch of other entertaining content, like this easter-egg map of how an architect conceives of the universe. (I'm surprised it doesn't have phenomenology in it, but instead it gets the more relevant "anomie.")

But the order form really made my night.

Motek, A Very Strange Software Company

I just read this article in the in-flight magazine on my way home from a short trip to Scotland: American Way Magazine: The Best Company to Work for in the World-Period.

As someone who needs at least a 3 day intro to a vacation to even start to relax, the European-style 5 weeks of vacation initially caught my eye and kept me reading. The rest of the story brought tears to my eyes (and I read a fair amount of business overview articles, for your average UI designer). A woman-owned company, with acknowledged lower salaries and internally published salary levels, in return for real quality of life outside the office? But that's not all.

Employees also know when an employee isn’t able to keep up with the workload. The result? Price offers a financial reward to employees who ask for help in order to stay on schedule. “The goal is to get the work done, not establish a star system,” she says.

Of course, it’s one thing to conjure up a cutting-edge culture and quite another to thrive amid relentless daily pressures. But Price hasn’t overlooked this aspect, either. While other companies talk about collaboration, Motek lives it. The company keeps a single to-do list... so that everyone is on the same page about priorities and the state of various projects. Anyone can enter an item, including customers and vendors. The list can include everything from ordering ink cartridges to customizing a specific function for a customer. Motek divvies up the tasks at meetings — and teams don’t pay any attention to who entered particular items.

The way Price sees it, the company’s success is a direct result of the money it invests in employees and of its commitment to developing a business structure that fosters knowledge-sharing and mutual goal-­setting. What’s more, “It’s amazing what a rejuvenated person brings back to the office, not just in terms of great new ideas but also in terms of enthusiasm and desire,” she explains. “The software industry’s idea that employees are entirely replaceable is absurd.” In fact, she believes it is precisely because of long hours and poor working conditions that today’s software is so buggy and behind schedule.

Apart from mourning that they are located in Beverly Hills (although I really do love my current job), it's now my goal to find other studies supporting her assertions. Let me know if you know of any...

Here's Motek's News page with links to many other articles about them.

Projects: Succeeded, "Challenged," Failed

The Standish Group have done numerous studies of software project success and failure rates over the last 10 years. They've observed some trends in project management suggesting things are marginally improving out there.

In 1995, their report identifies these key factors and associated weights contributing to project success:

SUCCESS CRITERIA	POINTS
1. User Involvement	19
2. Executive Management Support	16
3. Clear Statement of Requirements	15
4. Proper Planning	11
5. Realistic Expectations	10
6. Smaller Project Milestones	9
7. Competent Staff	8
8. Ownership	6
9. Clear Vision & Objectives	3
10. Hard-Working, Focused Staff	3
TOTAL	100

As you can see in the graphic below, in 1994 the the success rate for projects was only 16%, while "challenged" projects (over time and cost) accounted for 53%, and failed (canceled before completion) accounted for 31% of all projects. In 1998, 26% were successful. Then in 2004, we reached a 29% success rate. Project costs are higher and the success rates are lower for larger companies.

The 1995 piece has a scorecard to apply to your project to help you assess your potential for success. They're good questions (but a little intimidating even in 2006).

The 1999 report is quite sobering; they suggest that project size, team size, and project duration are the major factors correlated with success. They identify project management and standard infrastructure as crucial factors for team success. (Their comments on project management are well worth looking at, too, including the role of mentoring project managers.) I am quite sure that Scott Berkun would agree with them.

In 2001's report, they state that executive sponsorship replaced user involvement as the number one critical factor, because the IT world had caught on to the user involvement philosophy. Staunch business and management support now trump requirements analysis and user review of product design.

Finally, here is a telling quote about written communication on projects, from their original article:

Achieving the answers to solving project failure often lies in developing written communication such as problem statements, project plans, and detail specifications. However, one of the problems with any written communication is the participant's (reader's) level of understanding. As technologists, we think, write, and talk in a manner that is not readily grasped by many people outside our industry. Aside from sounding intimidating, you run the danger of the reader actually thinking they understand what you are saying, while your meaning may in fact be entirely different. To paraphrase the words of the English poet, Samuel Taylor Coleridge "Until you understand a reader's ignorance, presume yourself ignorant of his understanding". In other words, write the document devoid of all technical terms and pseudo technical terms. This includes words used by our industry, but rarely used outside our industry. Words like paradigm, metric, abstraction, and orthogonal, should not be used in any document if you want the normal reader to understand. Remember it is your job make the reader understand the plan. It is not your job to show how smart you are or to demonstrate that you can use big words.

Sadly, "phenomenological" is probably not a good word to put in a spec either.

Sleep on it for complex decisions.

Complex decisions are made better after a night's sleep, according to BBC NEWS: Sleep on it. It looks like simple decisions are better made consciously than unconsciously, though. (Now, how can you tell the two apart? And why does the alert brain seem so limited-- is this another "7 plus or minus 2" result?)

The conscious thought group managed to pick the best car based on four aspects around 55% of the time, while the unconscious thought group only chose the right one 40% of the time.

But when the experiment was made more complex by bringing in 12 attributes to weigh up, the conscious thought group's success rate fell to around 23% as opposed to nearly 60% for the unconscious thought group.

Importantly, post-decision satisfaction ratings were included in this study. So -- if you have a group of people in charge of making complex multivariate decisions, who also have sleep disorders, your organization is in trouble--? Unless someone up above gets a lot of sleep on all the complex decisions everyone is coping with, perhaps.

That question cuts a little close to home, for some of us. It also begs a new candidate interview question...

Secret Passages, "Ruse and Escalade"

Off steve: HiddenPassageway.com, an architectural folly company offering you built-in secrete passages and hidden niches appropriate to store guns in, according to one of their animations. The videos and small 3d animations are very cute.

Which site made me think of one of my favorite news stories from my time in France: The mystery of the hilltop monastery, the locked room and the missing manuscripts. A classic locked room mystery:

From that date, a succession of immensely valuable works, including precious early religious texts and several dozen heavy 15th-century illuminated manuscripts bound in wood and leather, began disappearing from the abbey's first-floor library. Police were flummoxed. "It was one of those frustrating but also rather thrilling cases," Madeleine Simoncello, the Saverne public prosecutor, said yesterday. "Quite extraordinary items were vanishing, sometimes singly, sometimes by the dozen. By last weekend over 1,000 had gone, yet the room wasn't even open to the public and as far as we knew nobody could get in."

The resolution of the mystery is better reported in this followup article after sentencing:

Gosse, a teacher at a Strasbourg engineering school and a former naval officer, faced a rare charge of "burglary by ruse and escalade", a reference to the tortuous climb in and out of the locked library.

He had found the route after discovering a forgotten map in public archives which revealed the secret access from the monastery attic. The map was a key exhibit in the trial. The attic, reached by a daring climb up exterior walls, led to a steep, narrow stairway and then the secret chamber. A hidden mechanism opened up the back of one of five cupboards in the library. The plans suggested that the secret route to the library, once the monastery's common room, served in medieval times to spy on the monks' conversations.

You can't make this stuff up. It needs to become a movie. To round out the fun, here's the wikipedia entry on secret passages, which points to an article from Jan 2006 about an enormous secret passage across the Mexican border to the USA. And finally, I found this fascinating archive of the Museum Security Network mailing list, full of tasty and weird stories of stolen antiquities.

Office Networks in Business Week

I've been catching up on some social network blogs, and have pointers to 2 short pieces I enjoyed in Business Week Online: IBM: Untangling Office Connections and The Office Chart That Really Counts.

In the former article, Kate Ehrlich at IBM Research notes that you want weak ties to fuel innovation, but strong ties to execute on it.

I think of the process of innovation as having three different phases: One is the generation of the ideas, and that's where you really need to have, not everybody in a group, but a few people who are connected to people who are outside. That's where you get these real "Wow, I never thought that something from that domain could be applied to this domain." So when you're talking about innovation, you've got to have somebody who can bring in the ideas from the outside.

In the second phase, you need to connect those people with others in the organization who are translators, who can say, "I can see that that idea might work in the company or group that you're in, but this is what we need in order to make it work for our product or our service or our process."

Then there's a third phase. You've translated it. You've got it set up. Then it's the delivery, and the delivery now is internally within the group. Now you want very strong ties between people. In network parlance, with innovation, you want weak ties, where people are more connected to those outside the group. [In the delivery phase,] you need strong ties in order to get the innovation executed.

In my more cynical moments, I question whether most businesses really want their employees innovating -- if they're not a research lab, that is. Most employees have too much ordinary work to do to take time to innovate and innovation from the ground up is hard to manage and control. But then I remember that finding a new way to do your work is innovation, too.

The latter piece has some excellent observations about the potential dangers of showing off who is connected to whom by social network diagrams. The same concerns apply to many visualizations of statistics in office life, of course.

For all of the benefits, charting informal networks can be disruptive. "Leaders feel pretty threatened by this," says Katzenbach principal Zia Khan, speaking of people who hold high perches on the organization chart but are more isolated on the informal map... It's O.K. for some people, such as those who spend a lot of time with customers or have expertise in niche areas, to show up on the periphery of the web. Maps can also highlight which employees might be too connected and therefore a potential bottleneck.

Confidentiality is also a touchy issue. A map that reveals who is well-connected and who is not can be destructive if it is shared too widely. "I know who I named, but when I look at the map, I might see [that person] didn't name me back," says Tracy Cox, director of enterprise integration at aerospace and defense contractor Raytheon Co. Now, says Cox, who does network analyses for the company's seven businesses, that hypothetical employee "knows that he is not valuable to his boss. And not only does he know it, but 50 of his closest friends know it, too."

This came up for me during my research days; and needless to say, it was an issue even if they didn't know who was A or B in the chart -- the map itself was frightening because everyone suspected they might be the X on the fringe.

Houston St., New York City.

Broken glass in New York City.

Is Your Boss a Psychopath?

At the risk of my boss reading this, here's an oldie but a goodie: Is Your Boss a Psychopath?

Corporate psychopaths score high on Factor 1, the "selfish, callous, and remorseless use of others" category. It includes eight traits: glibness and superficial charm; grandiose sense of self-worth; pathological lying; conning and manipulativeness; lack of remorse or guilt; shallow affect (i.e., a coldness covered up by dramatic emotional displays that are actually playacting); callousness and lack of empathy; and the failure to accept responsibility for one's own actions. Sound like anyone you know? (Corporate psychopaths score only low to moderate on Factor 2, which pinpoints "chronically unstable, antisocial, and socially deviant lifestyle," the hallmarks of people who wind up in jail for rougher crimes than creative accounting.)

A good point is made in that some psychopaths are created, not born that way, and American individualistic corporate culture is a petri dish for this type of behavior. They get ahead in part because of their traits.

Of course, cynics might say that it can be an advantage to lack a conscience. That's probably why major investors installed Dunlap as the CEO of Sunbeam: He had no qualms about decimating the workforce to impress Wall Street. One reason outside executives get brought into troubled companies is that they lack the emotional stake in either the enterprise or its people. It's easier for them to act callously and remorselessly, which is exactly what their backers want.

The good news is that a productive narcissist makes a great balance for an executive psychopath. Or a productive obsessive. It's all just one big mental hospital in a large corporation. (See the related NY Times short on presidents with mental illness symptoms.)

The Dumpster (Happy Valentine's Day!)

Off Information Aesthetics, the Dumpster is a visualization of breakups described on blogs in 2005, with sample color commentary from the posts. It's very pretty, and of course it was made in processing.

Joel on Interviewing, Me on Performance.

Joel Spolsky's latest post is on interviewing interns for Fog Creek. I love how seriously he takes this, but with the same caveats he quotes as getting from other people. Bravo that last year's interns wrote their fastest growing business product almost by themselves. That's a good use of interns! (And in an industry full of big software companies that just acquire existing products and can't build them from scratch anymore, it's refreshing.)

Joel's main goal in hiring is to get "Smart people, who get things done." (See his Guerrilla Guide to Interviewing if you haven't read it.) This makes much sense to me. Operationalizing it during interviews will mean different things to different teams and disciplines, I think. His techniques for screening developers aren't exactly the ones I'd use on UI designers, but they're not too far off. (I like giving design problems, asking for design evaluation of existing products or mockups, examples of work process and deliverables, and willingness and ability to do the boring and hard stuff in order to push work through to completion. Checking ability and interest in learning is part of this last item.)

I was talking to a friend recently about how academic journal and conference reviews are "how professionals enact their discipline," by setting up the boundaries they want on what counts as quality work and what is worth letting in to extend or challenge their field. (It takes a brave reviewer, even in a cross-disciplinary field like HCI, to recognize something from a very different perspective as a valuable contribution to the discipline.) The same could be said for the hiring process, although far less abstractly -- and it's a little more tactical. Your target processes are always going to be moving-- hopefully improving-- but you have to hire for short term success as well as investing in the future.

Hiring is hard in part because most folks aren't good at articulating what needs to get done, and how to evaluate people for those abilities. So often the emphasis ends up on wishy-washy and dangerous "cultural fit" terms, because the objective measures are missing. Performance reviews, which are obviously (or maybe not!) related to the hiring problem, are prone to the same failures; a lack of understanding of what needs to be done in an organization can lead to subjective personality measures instead of objective measures of what got done and how and what we need to do next year. (I sometimes wonder: what if, instead of the traditional performance review methods HR depts foist on us, we were to do the interview process all over again -- with a focus on what got done in the last year, instead of previous company work? No, I don't mean we fire people who get thumbs down; but we evaluate concrete work as if we hadn't spent the year with them disagreeing or drinking after hours and learning about their cheating ways or charity contributions.)

It makes sense to me that in organizations with a poor understanding of what their business is, with an incomplete understanding of how to achieve and how to measure success, you'll also find questionable (and confused) hiring and equally poor performance review processes. But in companies like Joel's that know how to measure success, hiring, retaining, and evaluation of employees is downright rational.

TopCoder's Programming Contest

The WSJ has an article on the recent Topcoder programming contest. And Fast Company did an article on the 2004 winner.

I've blogged pointers to the MATLAB programming contest before. An interesting difference (if I read this right) is that a major focus on the TopCode contest seems to be challenging other people's solutions and invalidating them with scenarios they haven't thought of, while an important aspect of the MATLAB contest is the highly likely possibility of winning by improving on other people's solutions. The two contests are Darwinian, but in different ways.

Davos Quotes on Creativity

Fast Company Now blogged about events at the World Economic Forum at Davos, especially the themes on creativity and innovation. The pieces are all quite short, but a couple are relevant to my last post on the NSF workshop on creativity tools.

In the "Innovation and Design Strategy" session, the panel members were asked to describe what they saw as the key to creativity.

Last to be voted off the island was Ideo's Tim Brown, who suggested that creativity is spurred by approaching problems with a beginner's mindset, and by exploring ideas through the use of rapid prototyping.

And the winner is: Google's Marissa Mayer, who argued for "a healthy disrespect for the impossible" combined with the virtues of constraints. In other words, aim high, but focus. Mayer described how an artist friend once told her that it was much easier to paint on a canvas that already had something on it--a mark or a line of some sort--than to begin with an entirely blank canvas. The existing mark is a constraint, something the artist has to think about and work around. And product developers at Ikea begin with a different sort of constraint, she said. They start with a price they have to meet--say $49--and then think about what they can make for that price.

I remember once being asked to just go off and design a solution for a big hairy problem -- with no realistic inputs at all to ground it or steer my direction. What's the ideal solution, Lynn?? I could have done something, but that waste of my time would have been more than I could have tolerated after the fact. The real development world is a world of constraints, and given a million other things I was also responsible for, I would have been in a bad mood indeed when my creative exercise was shot down as unrealistic. To this day, I am careful to always include minimal acceptable versions of every design idea I come up with -- because everything gets cut down eventually.

And the blank canvas is a bitch to start from, as she said. Translating that into software tool terms, I think it's more fun to start from something not quite right (a template for a document design, a building component?) and then modify it to get what you want.

Here's another one from their coverage, just because it's pithy and reasonable and I was on the page:

A Columbia University economist, Xavier Sala-i-Martin, spoke at a session on global competitiveness in Davos this morning. He offered what I think is the most succinct statement of the stages economies move through on their way to becoming innovation-based. First, he said, you concentrate on making something cheaper than anybody else. And when you can no longer make something cheaper than anybody else, you concentrate on making something better than anybody else. And when you can no longer make something better than anybody else, you concentrate on making something different than anybody else. That's the innovation economy.

Creativity Support Tools

Ben Shneiderman has posted the individual papers and extensive summary of the NSF-sponsored Workshop on Creativity Support Tools that took place in June 2005.

I haven't read all of them yet, but my initial impression is some disappointment that the workshop participants were academics and no one purely from the creative tool "user" persuasion was there. It looks a bit industry internal, and not just software industry internal, but research-world internal. Many of the literature citations support this, although I did find a book or two I didn't know about (e.g., Polya's How to Solve It). It wasn't surprising to see quotes from Csikszentmihalyi's Creativity but it was not clear to me that the cultural recognition of creative work as valuable should be relevant to the design of software to support the act of creating.

This said, there are interesting points, especially if you work on software design for creatives, as I have in the last 3 jobs (if you include scientists, which Mihaly Csik. does). The Design Principles article includes these items:

• Support Exploration: First, it must be very easy to try things out, and then backtrack when unsuccessful. This means that the tools must be trustworthy so that users are comfortable trying things. For example, a very good Undo capability is required in the tools.... A second requirement is that the tools be “self-revealing” so that it is clear to users what can be done. If the flexibility is not apparent, it will not be used....Another way to support exploration is to make it very fast to “sketch” out different alternatives at the early stages of design.
• Have Low Threshold, High Ceiling, and Wide Walls:Effective tool designs should make it easy for novices to get started (low threshold) but also possible for experts to work on increasingly sophisticated projects (high ceiling) [Myers 2000]. The low threshold means that the interface should not be intimidating, and should give users immediate confidence that they can succeed. The high ceiling means that the tools are powerful and can create sophisticated, complete solutions. Too often tools that enable creative thinking may be quite hard to learn (they don’t have a low threshold). Instead, they focus on providing numerous powerful features so that experts can assemble results quickly. Now, we add a third goal: wide walls. That is, creativity support tools should support and suggest a wide range of explorations.
• Support Many Paths and Many Styles. People work in different ways, right brain vs. left brain, "hard" vs. "soft" approaches.
• Support Collaboration among multiple users. In all of our projects, in schools, and in the “real world,” most creative work is done in teams.
• Support Open Interchange: The creative process will not usually be supported by a single tool, but rather will require that the user orchestrate a variety of tools each of which supports part of the task. Creativity support tools should seamlessly interoperate with other tools. This includes the ability to easily import and export data from conventional tools such as spreadsheets, word processors and data analysis tools, and also with other creativity support tools. This requires that the data formats in the files be open and well-defined.
• Make It As Simple As Possible - and Maybe Even Simpler: A plea to reduce featuritis, which will appeal to my boss, for sure.
• Choose Black Boxes Carefully:In designing creativity support tools, one of the most important decisions is the choice of the “primitive elements” that users will manipulate. This choice determines, to a large extent, what ideas users can explore with the tool – and what ideas remain hidden from view.

While the principles look good to me, the examples surrounding them are a little thin, as far as my needs go. But I found it interesting reading nevertheless and recommend browsing the site.