Putting PR research in its place

Ben Goldacre‘s Bad Science column frequently exposes less-than-stellar research findings that have been subjected upon the general populace. He’s always worth reading, and in the past has even inspired me to vent at some of the ridiculous claims.

But his latest column on PR reviewed data has, surprisingly, caused me to reconsider my stance.

In it, he compares the recent advertorials that the Express ran (which were outside of the rules) to these sorts of surveys. This led to a response from one of the main perpetrators  – One Poll – and they (sort of) agreed.

In their own words, “We’ve been providing branded, stat-based news copy to the nationals for more than ten years now. Why do you think we do it? Everyone is aware this is a branding exercise…”

Unlike the advertorial row, the transaction is non-monetary and thus legal. The journalist is essentially outsourcing part of their responsibility. In some ways, it is like a landlord taking on a lodger, with the lodger earning their keep through “chores” rather than paying rent.

Does it matter? I don’t disagree with One Poll when they say one of these survey stories can be entertaining and provoke discussion – though I might replace the word “entertaining” with “momentarily diverting”.

But I disagree when they say that the surveys are valid. In terms of validity, I would subjectively rank this approach as thus (moving from invalid to valid):

  • Made up data
  • Straw poll of close friends
  • Accumulating opinion from one source (e.g. comments from one news story)
  • PR survey
  • Hypothesis testing survey
  • Exploratory survey
  • Census

I have issues with their validity not so much because of the financial motivations of respondents, but because of an obvious inability to replicate the answers. They don’t have an objective truth.

To use one of their most recent press releases as an example: Man City fans are among the poorest in Britain. If the survey were repeated tomorrow, I wouldn’t be at all surprised to see a totally different result.

SIDENOTE: This post isn’t meant to disparage this sort of research (I’ve done that already), but for supposed experts they can surely create better copy than ”It’s no surprise to see Chelsea up at the top and the other big London clubs. They have a loyal ban base with supporters around the country and obviously have some money to spend.” Aside from spelling and grammar, perhaps they should have made the correlation between geographic disparity and affluence a bit more explicit…

While I still think this approach (to both research and marketing) is nonsense, I’m no longer against it appearing in newspapers. What I would like to see, however, is an explicit admission that it is a one-off accumulation of non-representative opinion at a single point in time (or something slightly catchier). It is not fact that Man City fans are among the poorest – it is just the result from a single, dodgy survey among people that live on the Money Saving Expert forums. These surveys are views, not news.

And with that caveat, I have no issue with this method. While labeled as market research, it is not something I, nor my company, would participate in and so we aren’t in competition with these practitioners. There is evidently a market for this sort of product, so good luck to those pursuing it.

One Poll have worked with some big names so they must be doing a good job in their niche. That niche is nicely summed up by their “No coverage no fee” results-driven model. The legal equivalents have been stigmatised as “ambulance chasers” – I wonder whether this type of service can avoid a similar slur…

sk

Image credit: http://www.flickr.com/photos/daveknapik/

PS I’d be interested in hearing from anyone who has commissioned a PR survey of this nature. Outside of the value of the column inches being greater than the cost of the survey, has there been any tangible, noticeable benefit to engaging in this sort of activity?

Reblog this post [with Zemanta]
Advertisements

5 Responses

  1. I haven’t commissioned it but I’ve spun a few bits of panel data into similar stories before.

    The world of social media seems particularly prone to this kind of survey-driven story, perhaps because they’re designed to be discussion-provoking (i.e. commentable, spreadable). Maybe the more blog-led a news sector is the more it relies on this kind of thing…?

  2. Excellent, excellent post!

  3. This relates to a questionnaire I am writing where responses will be tagged to a database – ergo its not research but CRM. It would be useful to have a glossary of terms agreed by trade bodies: fact find, audit, poll, survey should be given clear definitions so it is clear what is NOT research but something else and also the level of confidence it possesses. It couldn’t be enforced. But at least we would be able to comment that the football ‘survey’ was an online poll or somesuch which would have clear connotations for reliability

  4. Hi all – thanks for the comments and tweets.

    As far as I can make out, One Poll et al aren’t MRS endorsed so as long as they work within the Data Protection act they are pretty much fine to do as they please. It would be nice to have some sort of kitemark or signifier that a piece of information is robust though.

    My discomfort with it is not that research is story driven (I’ve worked on ad effectiveness studies – the research is meant to prove the story); it is just that the story is so arbitrary and meaningless. They are provocations with little basis (so in many ways similar to blogs or columns).

    Cheers
    Simon

  5. Interesting post Si. I’ve dreamed up and commissioned a few of these survey stories myself for work, so I’d be interested in having a chat with you about it sometime…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: