Marketers all spin numbers. That's what numbers are for. They're there to prove a point. So we try to make them prove our point however we can. But usually the goal is to at least make our interpretation of the numbers seem legitimate and unbiased. We try to create some measure of defensibility in our presentation. And that's just not the case in the latest Feed Report from Razorfish. Entitled, The Razorfish Digital Brand Experience Report, 2009, the study attempts to look at how the face of the consumer has changed. And yes, it's spun. Go figure. My trouble with it, though, is they could not have been more blatantly obvious that they came up with their conclusions before they even launched the study.
Ben Kunz, over at Mediassociates and blogger at Thought Gadgets, first alerted me that there were problem with the report, so I took a look. And while I wasn't nearly as fired up as Ben got, I had to admit that everything from the graphic representations and headlines to the way the findings were presented were overtly misleading.
Here's how I put it in an email to David J. Deal, Vice President of Marketing at Razorfish:
==
Like I said in my tweet, I realize these reports are generated to gain business, so they're spun to prove points. That goes without saying and all's fair in love and war.
But I think you got off to the wrong start with this. One of the first elements is a propaganda graphic about the end of the 30-second TV spot, which is so laughably biased it immediately put me on guard. Then you immediately launch into a picture of "consumers" before you define that this is a "connected consumer," so again the report establishes bias ahead of facts. And throughout the rest of the piece the graphics over-hype the value of digital at the expense of other media.
Understand, I am not a TV/Brand-Ad guy. I actually believe in a digital future and actively work to engage my clients in all these areas. But even I was turned off by the bias. That's saying something.
All that aside, though, I'm not sure how you reached several of your conclusions. For instance, how did you conclude that the connected consumer is the new median? And I'm not talking web use here (because clearly most of the US is connected), but the "connected consumer" as you defined in the report. There's a big jump of logic there to say that the group you describe in the report is representative of a new median.
Case in point, we can say that much of America has a DVR. But concluding that all users will now skip ads because the most engaged users do so is clearly false. Most (70%) just sit through the spots. [NOTE: I was wrong here. The statistic is closer to half watch commercials. But my point is still valid.] With this in mind, I question your proposed median. The level of engagement for your study group may or may not be representative of the median. The data is unclear.
Also, saying that people want to engage brands online and using as proof discounts strategies is a bit disingenuous. This equally proves that people will buy newspapers or watch a TV commercial if they know they can get an offer. And really, it's not so much proof of engagement as using digital mediums as another push channel. Engagement is building advocacy, which has little to do with commodity tactics like couponing.
Let me stress that the reason I'm being so critical is that I think the spin has done the value of this report a disservice. I think the data you did present is extremely rich and useful. It proves a lot of things of note and will help me tremendously. But it's all wrapped in statements that are seemingly unsupported and are bound to make people question the report in its entirety.
==
And really, my last point is the most important. I expect a report like this to be spun, so I don't mind a little hubris. But that hubris has to be at least defensible by the data presented in the report. Most of the conclusions here are framed as being proven in conjuncture with other reports that are referenced, but not seen. For me, that's not compelling.
Further, while we can dismiss this report for bias, it's high profile nature serves to make all reports like it subject to additional skepticism. And that just sets us all back. As Ben said in an email to me on the side, "If you really believe something works, and have a plan for doing it, why not have the balls to just go do it without worrying that 97% of the world agrees with you?"
Thank you to David Deal for engaging me on these issues and listening to my complaints. My intent is not to bash Razorfish and they have proven themselves to be engaged listeners to my criticisms.
UPDATE: David Deal contacted me via email today with some insights on why the Razorfish FEED report was constructed as it was, as well as some clarifications about how they arrived at their conclusions. I asked if I could reprint his reply and he granted me his permission:
"...thank you for your thoughts . . . here are a few observations . . .
The FEED illustrations are meant to capture popular sentiments about consumer engagement and advertising. As you can see, we had a little fun with them. The caricatures and general tone of the graphics are meant to impart a bit of humor and are not meant to be taken literally. (In fact, we produced a TV ad for All detergent earlier this year, which we linked to an online promotion.) For instance, the illustration on Page 16 with the police officer is intended to provide a little levity amid all the attention paid to consumer feedback mechanisms like Yelp. However, I’m glad you told me about how they struck you – sorry that they got the report off on the wrong foot for you!
FEED asserts that connected consumers are becoming the new mainstream based on the research such as the Forrester data cited on Page 17 and the “digital fluency” data cited on pp. 18-21. Moreover, for our clients, connected consumers are indeed the new mainstream. This cohort spends money online and is quite active online . . . representing an important audience to marketers. That said, I think we can be more clear next time about what we mean by this observation, especially the part about this cohort becoming the new mainstream for marketers, per se.
The observations about offering deals on Twitter and Facebook are quite interesting. I think there is another point that emerges now that I’ve thought about FEED over the past few days: consumers will respond to brands that make them feel special, whether through an offer of exclusive deals or content tailored to their needs. A Facebook fan page that makes them feel special one way or another will engage them over a period of time because consumers will keep coming back for more content.
Certainly there is no perfect research report, and we are always looking for ways to improve – so thank you for taking time to share those observations so that we can articulate our conclusions more clearly!"