Victoria's Secret Market Research? Or not.

From my huge backlog of things to post, today I choose Victoria's Secret's online survey -- because I'll be tickled by the effect on my web logs.

Yes, in the past I have bought from VS, and do still buy their bras from time to time, despite the price tag. So I got sent an online survey. I always take these market research surveys for professional reasons, since I write them myself for clients from time to time. This one was, well, just strange.

It's hard to judge it as bad or good without knowing what branching logic they have built in. Branching means: If a respondent picks option (a), show them a different followup set of questions than if they had picked option (b). So I may not have seen the whole thing, and may have ended up in a strange cul-de-sac for people who buy bras because they're sexy. I hope not.

When I got to this checklist of "how does your bra makes you feel" (or somesuch), I was genuinely surprised. There are no negatives in here, and the word "supported" doesn't appear. I wonder what they can learn from this, apart from what they want to hear? The only way to avoid their positive bias is to check nothing, which I suspect will be tough for that helpfully-minded customer set that like to fill out surveys.

Previous to this question, they asked about other retailers you buy from. Now, if they had the usual sort of market research plan, I would expect to see an attempt at a basic SWOT analysis on the results: analysis of their strengths, weaknesses, opportunities, and threats.

How do you do that? Well, the logic is roughly this:

1. Ask:Where do you buy your bras?
2. Ask:What's important to you about your bra?
3. Ask:What's your feeling about the bra you bought?
4. Data analysis: For people who bought from us in (1), what's the difference between (2) and (3). If there's a big gap, that's our opportunity, going forward. (And roughly, strengths and weaknesses, when you compare against people who bought from Sears and Macy's and Frederick's of Hollywood on the same dimensions.)

If they're going after pure brand image and evaluation of their own success at achieving it, I think they still screwed up. Or at least they lost a serious opportunity in their data collection. They know where I shop (or where I said I shop, where I remembered I shopped), and they didn't have any adjectives that don't seem to be their personal target brand image in the list.

With the right vocabulary choices, including negatives and neutrals, they could have done some interesting segmentation based on where their own customers fall in "feeling" versus the other retailers' customers. Something like this:

They can always conclude from their data that some of their customers aren't that interesting to them from a marketing perspective (e.g., the people who shop at Sears, like their stuff a lot, and only occasionally go into a VS store -- because they'll be hard to capture if they're not dissatisfied enough with Sears).

In any case, some other common mistakes I see in market research via survey:

• You try to collect too much in one survey, and can't do the analysis (plus you irritate the people who filled it out). You can always get more data later in other forms.
• You don't know what you're trying to get out of it, so you can't construct the instrument well to get anything at all.
• You set it up to learn what you want to hear (which is what I think was going on with VS -- I won't even tell you about the underwear questions), so you learn nothing and waste time, money, and your customers' patience.
• You collect the data, then don't know what to do with it. You either don't do much, because of lack of skills in data analysis (clustering, mining, etc), or worse, you do nothing. You have it, but didn't take advantage of it. (This makes me quite uptight when I run into it. Good data is gold. My consulting tagline is "data-driven" for a reason.)
• You solicit data from the wrong people, but don't even know it, because your survey didn't check on their credentials for answering and providing input. So you can't toss out any of the responses to clean the rest of the data.

End data rant of the week... I think I'm going to Sears now.