Atlas Mountains Town

This is where they filmed the slave town in Africa for Gladiator -- they added a fake Hollywood gate to the front of this place for the movie.

Google Patents Sketches

If you like sketches of product designs, and things people patent (often very odd), I recommend Google Patents search. For the product designer, these sketches are often educational as effective communication tools.

Warning: Most software companies don't want you doing any patent searches on anything you work on, because it can get you in legal trouble if you have prior knowledge of existing patents when you produce new designs. So avoid searching on patents related to your current software design work!

Dennis Frailey: A Day in the Life

Last week I was lucky enough to be in a 3-day training course on project management. I was not entirely sure this course was a good idea for timing and content reasons, despite having a great interest in the topic myself-- but the instructor proved me wrong.

Rather than the usual consulting company instructor (these people often piss me off), we got a thoughtful, wise practitioner, who impressed all of us (a tough crowd) with his depth of experience in the software industry and his thorough research understanding of the problems we were facing.

Dennis Frailey's ACM "a day in the life" bio story is here. The university classes he teaches are described here. He has made career changes based on revised understandings of the "real" problems he encounters in daily work. I share his impressions about research and practice, although I've had far less experience than he has at either:

Although I loved teaching (and still do it on a part time basis), I quickly learned that survival in academia is based on "research", not education. And the numbers game rewards you for cranking out large numbers of papers rather than small numbers of really good papers. I did some innovative research in the areas of real time operating systems and computer architecture, but I found most of the really interesting work in these fields was being done in industry... And when I was granted tenure, I learned that the research that counted the most was what I had done for industry and could not have done in academia. Moreover, there is a certain sense of satisfaction one gets from building a real product that one cannot get by writing research papers. This led me to consider and ultimately to accept a permanent move to industry, which I did in 1977.

Once in industry, at the corporate engineering center of Texas Instruments (now part of Raytheon), I discovered something else. Most of the real problems are not to be found in the research labs but in the areas where real products are being planned and developed. Thus I migrated from doing computer architecture research to actually building computer and software systems that had to work. This gave me a deeper appreciation of the intellectual challenges and rewards associated with "getting dirt under your fingernails" so to speak. It also gave me a totally different understanding of what software "engineering" is... Once I had to make reliable systems that peoples' lives depend on, I began to appreciate the need for "engineering discipline" and for greater emphasis on understanding the processes we use to develop software. This led me to move into new parts of the company where I learned about different kinds of issues. Many years later, my varied background has qualified me for a senior technical position where I am expected to understand the entire scope of a problem, from the technical details to the management concerns.

I bailed from research for some similar reasons, during the dot com boom. I've also had the intuition for years that the hardest problems in building software are not the technical problems or even the design problems, but the management problems. Which may be one reason that UI design and usability methods haven't had the impact they should have had on the industry -- not because of lack of the value they add, but because lack of organizational historical data (of most kinds, but especially quality and process-related), management incomprehension and lack of good design and planning during the dev process, and resulting interdisciplinary confusion and contention when pressure hits. Not to mention that even without the design process differences that UI and usability introduce, complex software development management seems to be entirely lacking at many companies. Chaos, confusion, and conflict are the norms I've experienced in most schedule-driven releases. Few people can make the complicated, emotionally-laden tradeoffs required in a mature, big-picture style, because there just aren't enough managers with this kind of skill and vision.

You can tell a lot about someone from who they admire and why. Dennis's response to this question:

One man I admire is Eugene Helms, who was my first boss at Texas Instruments. He would often sit through a meeting, say very little except asking a few questions, and in the end would sum up the most important points - i.e., he listened, thought about what he was hearing, and put it all together. He also trusted those who worked for him to know more about their specialties than he did. As a result, he could see the big picture better than just about anyone else. He was kind and supportive, and he would stand up for what was right.

I admire Dennis, and I hope that says good things about me.

VPs as Indicators of Problems

Scott Berkun has a nice snarky post about VPs of Innovation; if you've got one, you're probably in trouble on that front, and it's not likely they're going to help.

Which reminds me of a company in my neck of the woods (not mine): they're created a "VP of Retention." I heard this from a friend recently, as we've interviewed a bunch of folks bailing ship from that place. Ok, kudos to their executive staff for noticing they have a problem; but sad and sorry it is that they had to "solve" it at that level, didn't notice before, and haven't done enough analysis internally to understand why they have a management disaster on their hands.

Management is the least recognized role and the most important, in these situations, in my experience.

A collection of professional essays...

The more involved in management I am, the less I feel I am doing things that I can point to and say "I did that." (Also, the less I work for companies that make consumer or well-known products, but that's a different issue and a weird trend I noticed a few years ago...) But I got the good news today that an essay of mine will appear in a book accepted by MIT Press for 2007 publication along with many others by names that I know from my field: luminaries, old friends, and former research colleagues. Most of the other authors are probably wondering who I am in this list; I feel honored to be there and to have been invited to submit something to it. Here's the info on the current Table of Contents: HCI Remixed: Essays on Works that have Influenced the HCI Community. (This book must be 500 pages long!)

We're all writing about previous work that influenced us, in a personal essay vein. It could become a nice secondary textbook in a reading course in interface design or human-computer interaction research. I was fascinated by the list of source papers and immediately downloaded a bunch that I didn't know myself. As soon as I flipped through them, I wanted to know what the other folks said about them. I can see the appeal of this collection immediately. Great idea, Tom and David.

Alligator

Data and Infovis and "Art"

I've been thinking about data a lot, since the Infovis 2006 symposium. At this conference was a strange mix of scientists, mathematicians, and a few artists, or those with an artistic bent.

My friends Martin Wattenberg and Fernanda Viegas from IBM Reseach Cambridge secured funding, invited submissions, reviewed, set up the equipment for, and then sat guard over (missing talks in which they were cited) an art show of infovis applications. They were specifically featuring artistic displays of real data (I'm paraphrasing what I think they said were their selection criteria. One was Golan Levin's The Dumpster, which I blogged about a while ago.)

To introduce this art show, they gave an excellent talk that I'd summarize as "What's Going On Out There in the Real World That You Might Not Know About." A bunch of us saw a lot of people in the audience noting down the existence of Ben Fry's Processing Toolkit that makes programming datavis apps accessible to artists and ordinary people who aren't postdocs in mathematics. Sadly, it reminded me of 5 or even 10 years ago when the CHI and CSCW research communities realized web startups had already made community apps that worked and they weren't made by researchers in labs. Where's the actual innovation happening? More often than not, it's students or other clever people with time on their hands and a willingness to play around.

But back to data: When I was doing my dissertation, data was a sticky subject. Collecting data on "human subjects" was overseen by strict board reviews and ethical examination, and I had to go through this as an early internet researcher with a Human Subjects Board who didn't know what to do with this kind of data.

The community I "studied" reacted strongly to some of the data that I collected, post-processed, analysed, and reported, regardless of the reviews I went through. My data said some things that they didn't want made visible, or suggested things they didn't like simply reducible to graphs and charts. (The book is available here, the last chapter discusses this problem in some detail.) Anyone who looks at or exposes recorded human behavior is going to hit this: for example, people who don't think they talk much and discover they talk all the time often don't like knowing this, however measurable it is and however potential this exposure might be for them. Which brings up the questi0n of why and when should you turn something into data? And analyse it?

So, thinking now about how the research and infovis worlds have evolved since then, and the new inevitability of data mining on behavior from the traces we leave behind us, I see these data source dimensions:

1. Data sets that exist and are known to exist-- census data, weather data, stock market data.
2. Data that "happens" but isn't necessarily assumed captured or turned into a set that's easily analysable: email, chat, mobile phone records, my retrievals from ATMs, where I walk and what I eat.
3. Data that we set out to measure, because we're looking for something: experimental data, NSA tapping us, etc.
4. Data we have (from any of the above means) and we converted to another form of data: e.g., turning activity logs into summaries of time on tasks, turning gene sequences into musical notes, turning video of your cats into a single overlayed image, turning text into images, etc.

The really creative apps for infovis often seem to lie in item 4), because transformation of data into other modalities is a trick of visualisation that might give us insights we didn't have before. Some of them are just elegant visualisations of data we wouldn't have thought of visualising (like Ben Fry's zipcode applet that Martin called an infovis "haiku"). The "insight" part is still tricky to handle; human perception differs, and reasoning skills differ, and that makes drawing conclusions from visualisations tricky too. (Untutored people generally make more of statistical tests than they should, too.)

Martin and Fernanda stayed safely away from defining "art" but I still thought about the artistic component of data mining. The value of data mining and the ability to form and then test hypotheses from different views of data is a skill, perhaps even an art in itself. An event occurs: I capture it, I capture multiple instances of it, and I look for patterns in different views of it, and then I learn from it or measure it some more or in another way to progress towards some truth.

Or, for the more artistic data visualiser: she captures it and events like it, she presents it in a novel and beautiful way, hopefully with some elegant interactivity, and other people learn something. The might learn something ineffable or impossible to reduce to words. But that doesn't make it less important. Scientific creativity still springs from the indescribable ideas you have about the world before proof and publishing.

Moroccan carpet seller

In Morocco. (It's tough to pick which ones to post -- I've got so many.)

Social Drinkers Earn More Money

Somewhat disturbing, but ringing true in a bunch of dimensions... This study shows that social drinkers earn more money than non-drinkers, and claims it's because of the increase in social capital gained by knocking one back with colleagues.

Although there is a united campaign to restrict alcohol, labor market data may surprise noneconomists: recent studies indicate that drinking and individual earnings are positively correlated. Instead of earning less money than nondrinkers, drinkers earn more. One explanation is that drinking improves physical health, which in turn affects earnings (Hamilton and Hamilton, 1997). We contend that there is an economic explanation. We hypothesize that drinking enhances social capital, which leads to superior market outcomes. Glaeser et al. (2000: 4) describe social capital as “a person's social characteristics, including social skills, charisma, and the size of his Rolodex, which enable him to reap market and nonmarket returns from interactions with others.” Some aspects of social capital might be innate, but people can enhance others, such as Rolodex size. If social drinking increases social capital, social drinking could also increase earnings. We attempt to test whether drinking enhances social capital by differentiating between social and nonsocial drinking; we predict that those who drink in public will have higher earnings than those who drink at home. New data confirm that drinkers earn more, and we find that social drinkers earn even more.

The article is here and comes with a somewhat scary libertarian slant intro, be warned:No Booze? You May Lose:Why Drinkers Earn More Money Than Nondrinkers (pdf). Note, this obviously supports the value of conference trip networking as important for career, if money is an indicator of career success (it is to some).

E3: Effective, Efficient, Elegant

Hi, my name is Lynn and I work all the time. Too much to post much these days.

But I had a nice break when I went to Infovis 2006, the symposium on information visualization. There was a bit too much math this year, starting from the keynote, which was Eades talking about graph layout algorithms. I still managed to get something thought-provoking from it. His criteria for algorithm evaluation was "effective, efficient, and elegant."

These are good principles for software design as well as algorithm design: a good piece of software should be effective at supporting the tasks it's designed for, be efficient in use and for use, and ideally is elegantly designed. Elegance, of course, implies more than "usability." Usability is a word that's got kind of an old school ugly lab study connotation these days; it's a word that doesn't say enough to capture current thinking about the value of delightful design, rather than just adequate design, in creating a differentiating user experience.

What's "elegant" in a proof or theorem, I asked of a friend who was a mathematician in his previous life. "Simple," was the first thing he said. But not just that -- it can be taught to a second year student, was one of Eades criteria (suggesting "learnability"). Yet also somehow "surprising." An elegant proof is a result with a twist you didn't see coming, but should have, adds an insight that makes it aesthetically pleasing.

At risk of triteness, I did look up elegance on dictionary.com after striking out in a Google search: "gracefully concise and simple; admirably succinct. Combining simplicity, power, and a certain ineffable grace of design." It adds:

The French aviator, adventurer, and author Antoine de Saint-Exup'ery, probably best known for his classic children's book "The Little Prince", was also an aircraft designer. He gave us perhaps the best definition of engineering elegance when he said "A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away."

E3 makes a good compound principle for evaluating design of all things, including software. I'd really like more elegance in my design.

10 most real life ghost photos (sic)

Ah, Pravda. It never fails to appear on the Anomalist. Here is a sample of photos of ghosts from the Russian online paper: 10 most real life ghost photos - Pravda.Ru.

Atlas Mountains at Sunset

One of my vacation nights involved a hike into the mountains.

Morocco, Atlas Mountains, October 2006

The Powerset Blogstorm

Can anyone beat Google? Well, the natural language processing community is having a go at it. I got back from 2 weeks of vacation to discover that an old friend and mentor from grad school days is now at startup Powerset, co-founded by Barney Pell, whom I knew as a grad student in Cambridge years before. There've been a rash of articles in the blogosphere about their announcements, as Barney summarises:

Barney Pell's Weblog: The Powerset Blogstorm: 1 week later.

Yet more evidence that startups are back and interesting again! And this time they want UX people earlier rather than later, which is a nice change for the industry.

Park near my house in Natick

Wetlands

Why Crunch Mode Doesn't Work

I have long been fascinated (and disgusted) by the lack of government, union, or HR care for the abuse of workers in the software industry. During the dot com era, I actually heard it touted as an intentional strategy by some Silicon Valley managers to hire young programmers right out of school who would work long hours for little pay and were considered "disposable." Mostly, they were disposed of, by burn out.

This attitude was reflected a little more indirectly in the cuts in benefits and pensions everywhere including major companies like AT&T, layoffs in which senior people were cut on the principle that they were easy to replace; among other people mismanagement decisions that seemed to me to be incalculably stupid. Incalculable, unfortunately, because the decisions and decisionmakers at these companies were protected by layers of indirection (about the reasoning, minimally) and because there is profound difficulty in measuring the damage to the company, products, and customers, both long and short term. Stupid, because the people in the trenches doing the work understood the impact every day, in terms of workload, quality of work, culture, their attachment to the company, belief in the industry and in what they were doing...

More than one person I knew who was young and abused in the dot com era decided they were "getting out," went to open a record store (or flower shop) and never looked at a computer again. In an era of declining enrollment in computer science programs, we as an industry can't really afford that.

Here's a guy in the game industry--one of the most famously abusive software sectors--trying to make the case with a fairly well-researched article on the subject: IGDA - Articles - Why Crunch Mode Doesn't Work: 6 Lessons. Tagline: There's a bottom-line reason most industries [except the software industry] gave up crunch mode over 75 years ago: It's the single most expensive way there is to get the work done. Of course, in the software industry, quality hasn't mattered that much, till recently. Will things change? Will software management change? Will the calculation become easier to make to argue against the stupid?

And here's an article, only tangentially related, but I think still related, on why America is lagging in R&D. The State of Research Isn't All That Grand. R&D is being outsourced, because it's cheaper that way; and R&D is "hard to measure" because it's a long-term investment. Most American companies aren't actually that good at long-term anything, in a profits-now-or-your-bonus-is-impacted business world.

Why hasn't Built to Last had more impact?

Birds at the Ren Faire in Carver, MA

Archimedes looking intense. RenFaire, MA, 06

Cheap Tickets

Over the weekend I caught up on Lifehacker, and found a bunch of posts and comment posts related to finding cheap airline tickets. Some were quite surprising to me -- like shop between 12 and 1am on Wed morning when companies update their records.

TripStalker -- a bot utility that continually looks for your best price and notifies you when it finds it.

Lifehacker thread of comments on this topic.

Another link collection on cheap ticket gimics.

Lots of people mentioned Sidestep (I use occasionally) and kayak.com, and travelocity doesn't fare too badly in the listings.

A fluffy photo post: Rabbits and Flickr Stuff

A friend sent me this link to some hilarious rabbit photos: My Rabbit Disapproves.

Just after laughing a lot at the justice in this woman's photo captions, I found this new flickr tool for browsing pic categories, which I spent a fair amount of time enjoying when I should have been driving to Nantucket. The search on bunnies turns up some really nice rabbit pics, not disapproving in the least. Cats are okay, but not as nice a collection. Gargoyles are a good contrast, and turn up a surprising number of cat pics. Or, perhaps not surprising, to a cat owner.

Flickr Storm.

Youth Hostel, Nantucket.

It's a good thing I'm not too old for youth hostels yet, or I would have missed this pretty building on the south side of the island.

Rusty Bench

Nantucket, MA