Putting PR research in its place

Ben Goldacre‘s Bad Science column frequently exposes less-than-stellar research findings that have been subjected upon the general populace. He’s always worth reading, and in the past has even inspired me to vent at some of the ridiculous claims.

But his latest column on PR reviewed data has, surprisingly, caused me to reconsider my stance.

In it, he compares the recent advertorials that the Express ran (which were outside of the rules) to these sorts of surveys. This led to a response from one of the main perpetrators  – One Poll – and they (sort of) agreed.

In their own words, “We’ve been providing branded, stat-based news copy to the nationals for more than ten years now. Why do you think we do it? Everyone is aware this is a branding exercise…”

Unlike the advertorial row, the transaction is non-monetary and thus legal. The journalist is essentially outsourcing part of their responsibility. In some ways, it is like a landlord taking on a lodger, with the lodger earning their keep through “chores” rather than paying rent.

Does it matter? I don’t disagree with One Poll when they say one of these survey stories can be entertaining and provoke discussion – though I might replace the word “entertaining” with “momentarily diverting”.

But I disagree when they say that the surveys are valid. In terms of validity, I would subjectively rank this approach as thus (moving from invalid to valid):

  • Made up data
  • Straw poll of close friends
  • Accumulating opinion from one source (e.g. comments from one news story)
  • PR survey
  • Hypothesis testing survey
  • Exploratory survey
  • Census

I have issues with their validity not so much because of the financial motivations of respondents, but because of an obvious inability to replicate the answers. They don’t have an objective truth.

To use one of their most recent press releases as an example: Man City fans are among the poorest in Britain. If the survey were repeated tomorrow, I wouldn’t be at all surprised to see a totally different result.

SIDENOTE: This post isn’t meant to disparage this sort of research (I’ve done that already), but for supposed experts they can surely create better copy than ”It’s no surprise to see Chelsea up at the top and the other big London clubs. They have a loyal ban base with supporters around the country and obviously have some money to spend.” Aside from spelling and grammar, perhaps they should have made the correlation between geographic disparity and affluence a bit more explicit…

While I still think this approach (to both research and marketing) is nonsense, I’m no longer against it appearing in newspapers. What I would like to see, however, is an explicit admission that it is a one-off accumulation of non-representative opinion at a single point in time (or something slightly catchier). It is not fact that Man City fans are among the poorest – it is just the result from a single, dodgy survey among people that live on the Money Saving Expert forums. These surveys are views, not news.

And with that caveat, I have no issue with this method. While labeled as market research, it is not something I, nor my company, would participate in and so we aren’t in competition with these practitioners. There is evidently a market for this sort of product, so good luck to those pursuing it.

One Poll have worked with some big names so they must be doing a good job in their niche. That niche is nicely summed up by their “No coverage no fee” results-driven model. The legal equivalents have been stigmatised as “ambulance chasers” – I wonder whether this type of service can avoid a similar slur…

sk

Image credit: http://www.flickr.com/photos/daveknapik/

PS I’d be interested in hearing from anyone who has commissioned a PR survey of this nature. Outside of the value of the column inches being greater than the cost of the survey, has there been any tangible, noticeable benefit to engaging in this sort of activity?

Reblog this post [with Zemanta]
Advertisements

Links – 23rd December 2008

Part 2 of the link update for December, and my final post of 2008 (barring unforeseen events).

Media channels

Scott Karp at publishing 2.0 channels Seth Godin with his call to arms for the print industry. The market and the internet don’t care if you make money, and the industry needs to adapt if it is to survive.

Futurescape have shared three of their excellent reports on web series, while the Observer looks at the successes of several of them. Check out my twelve shows to check out here

Grant McCracken wonders why TV revenues are holding while viewing declines. I haven’t seen the data he is quoting, but in the UK overall viewing is actually pretty robust (it is just fragmenting). I would also argue that TV is better suited to adapt to the new media landscape than radio or press, though I’m sure people from those respective industries would vehemently disagree.

The New York Times’ 8th annual Year in Ideas (some better than others)

A 25 point manifesto for the music industry

Music Ally has a load of predictions for digital music in 2009

Marketing and business

Apathy Sketchbook has accumulated a magnificently comprehensive list of all the terrible PR formulae masked as science. Harks back to my Bad Research post.

With ROI discussions threatening to jump the shark (if they haven’t already), everyone should read Lewis Green’s reminder of what ROI actually is, and how it differs from value.

Seth Godin asks when you create a new product or brand, are you making a new market or taking from an old one?

I’ve already linked to Gareth Kay’s excellent slideshare presentation, but this summary contains some great comments on the problems of planning.

Tom Peters has 27 practical ideas to transform your organisation

The Ad Freak awards for 2008.

Le’Nise Brothers has some great advice on digital media planning

The Advertising Lab has published 19 tips for in-game advertising

Brand Strategy has 9 tips for businesses in 2009

A MetaFilter thread on products where it is better to spend more on quality – can this advice still be adhered to in the current climate?

In a nice piece of bricks and mortar experiential marketing, P&G opened a store for its coupons on Black Friday.

Miscellaneous

The Big Picture is one of THE great web innovations by traditional media, and their year in pictures is a must

Foreign Policy again publish the ten stories you would have probably missed over the past year – which is shocking, given the importance of them

Malcolm Gladwell uses quarterbacks and teachers to ask why we hire people when we don’t know if they will succeed

12 fascinating and mysterious criminal cases does exactly what it says on the tin – includes Abe Lincoln and Lizzie Borden among others

The life of Carl Ponzi – after whom Ponzi Schemes (a form of pyramid selling) take their name

Hitotoki brings together literary tales of visits to specific parts of London.

Particular commendation goes to The market and the internet don’t care if you make money, Year in Ideas, terrible PR formulae masked as science, what ROI actually is, The Big Picture and ten stories you would have probably missed over the past year

That is me well and truly spent for the year. It’s been a blast. I hope everyone has a wonderful Christmas, and I’ll be back in 2009.

sk

Links – 13th October 2008

I’m away for the second half of this week so I am running two 10-day link updates, rather than have a gap later this week

(This is in no way related to me not having time to get this together last week)

Blog-related

Can marketing survive without the Grand Gesture? (Chroma Inc)

A great overview and analysis of Digg’s recent problems with crowdsourcing (Mashable). My post on crowdsourcing is here

A very thorough summary of the major players in the online conversation tracking market (Ryan*MacMillan)

Ben Goldacre highlights a form of Bad Science/research that is not only biased but borderline fraudulent. And ICM put their name to it. (Guardian). My post on bad science/research is here

Adrian Chan on why, in pursuit of the long tail, the power curve shouldn’t be overlooked (Gravity7)

John Battelle on why Google (and Google Maps) needs to add the human community element to their algorithms (Look Smart)

Nigel Hollis of Millward Brown defends the rational AIDA approach of the Link Test on his blog. He raises some valid points, but I remain slightly sceptical. Saying that, I cannot propose an alternative, cost-effective approach

Market Sentinel take a look at how to measure online campaigns. My post on online audience measurement is here

Seth Godin imparts some further PowerPoint tips

Random

Jay Walker has created the most amazing library ever – check out the photos on Wired

100 skills every man person should know (Popular Mechanics)

Rolling Stone offer a character assassination of John McCain – the level of partisanship in US media continue to shock me. Republicans may (from my viewpoint) go further below the belt, but Democrats aren’t angels. Irrespective of whether the bulk of this article is true, the underlying vicious tone turns me off

A slightly lop-sided split there but particular commendation to RMM London’s analysis, problems with crowdsourcing, Bad Science/research and the grand gesture

sk

Bad research: Compromising the value of PR

NB: The inspiration for this post is Ben Goldacre’s excellent Bad Science column in the Guardian. The book came out this week (here is the Amazon link), and the Guardian has serialised chapters on the MMR jab and miracle pills.

I particularly enjoy the columns where he remorselessly picks apart a PR piece containing some level of human interest, but one that is entirely based on spurious research. This deconstruction of Jessica Alba having the perfect wiggle is a fine example of the method.

A doctorate-for-hire is commissioned to create a formula out of thin air. The story generated then gets picked up by the press. The sponsor is named in the article, and they conveniently have some tenuous link to the subject. The PR pays back the research cost.

The most depressing thing about this endeavour is that there is no conclusion in sight – they just keep on coming.

I work in a Research & Insight department. More often than not – and I include myself in this – the insight is dropped and it is simply referred to as the research department.

This is fine, but it is important to note that the two functions are distinct. Insight is about finding new things out and threading pieces together to form fresh conclusions or intelligence. Research is providing evidence to support a theory or hypothesis.

(Note that these are my own personal definitions informed in the most part by the way in which my job operates. Wikipedia defines them differently)

Both are necessary, but both can be compromised. Problems with insights tend to be more innocent – flights of fancy where the new findings (intended or otherwise) don’t justify the expenditure invested in producing them.

Problems with research are more sinister. The answer is already known. The end truly justifies the means. The research design, the wording of the questions and the data cuts providing the analysis are contorted to ensure that the correct answer is given.

In reality, this doesn’t (well, shouldn’t) happen. To take a form of research I am familiar with; advertising effectiveness studies don’t always produce positive results. If results are bad, the client and agency are informed (albeit with any silver linings accentuated), and the study is swept under the carpet.

The major problem is when the research resembles, but doesn’t match, the pre-ordained conclusions. Then the temptation seems to be too great to resist. So, the results are tidied up. The supporting evidence is hidden behind hyperbolic headlines and the announcement is made.

For all intents and purposes, the evidence may as well be removed. It only gets in the way of a good story.

Canon – “world-leader in office imaging solutions” – recently came up with a doozy. As you may have been aware, the Beijing Olympics recently occurred. Did it inspire office workers to emulate their sporting heroes and get fit and healthy? Of course not, and Canon has data to support this claim. Apparently officer workers spend “the equivalent of a staggering 34 working weeks per year”.

Fortunately for us, “Canon has teamed up with health professionals from the fields of dietetics and ergonomics to develop an ‘Office Olympian’ guide. The guide includes independent expert advice on a range of topics such as keeping active in the office, healthy nutrition advice and perhaps most importantly, correct posture for employees who spend long periods working at a computer”

Phew!

And for just the cost of a few questions on an omnibus survey, press coverage is acquired.

But let’s take a closer look:

  • The 34 weeks number comes from office works spending, on average, five and a half hours of work per day at their desks. In what conceivable way is this staggering? On average, people only have 90 minutes worth of meetings and toilet breaks per day?
  • Time with friends and family, exercising and chores are sacrificed when two thirds of office workers work beyond their contracted hours. The frequency and length of this overtime isn’t elucidated upon. I can only assume it is regular and extensive
  • The work-life balance is destroyed because a fifth of workers spend 7-8 hours a day in the office. Because with the 2 hour each way commute and need to get 12 hours of sleep a night, there really is no time to have a life outside of work
  • 20% of workers don’t consider their health when in the office, despite spending the majority of their time there. Firstly, health is generally only considered when there are negative repercussions, so that would imply 4 in 5 are healthy. Secondly, how does five and a half hours a day for five days a week over 47 weeks a year constitute the majority of my time?
  • This is my favourite one; Only 19% use the tea run as an opportunity to take a break and just 28% regularly leave their desk to pick up documents from the printer – an ideal opportunity to stretch and exercise. See how Canon’s world class imaging solutions help improve your life? Because I don’t drink tea and have no need for my rubbish non-Canon printer, I have no need to leave my desk. Outside of the 90 minutes I spend away from it of course. I think Canon missed a trick here. Consider the downtime involved in going to the toilet – surely some stretches and exercises could be incorporated into that?
  • A couple of rent-a-quotes are then wheeled out for the coup de grâce.

Just writing the above has actually made me quite angry.

Perhaps I am too cynical? There are no doubt some good intentions burrowed beneath the marketing effort, and some people may genuinely gain benefit from the tips on diet and ergonomics.

But when the advice is packaged up in such a moronic fashion, it completely destroys any appeal that the campaign may have instilled.

At some point, either the press must resist publishing these “stories” or the sheer ridiculousness of their claims must be exposed. But in the meantime, there appears to be no respite.

I eagerly await the next release on anger management.

sk