Research vs Planning

Image credit: http://www.flickr.com/photos/darkumber/

Preamble

I’ve learned a lot from reading and interacting with various blogs. But something has been bothering me for a while now.

What if it is all complete rubbish?

That’s blatantly trolling and patently not true – I read a lot of fascinating and thought-provoking posts. But the extent to which I should implicitly trust their veracity is something that is concerning me.

There are definitely zeitgeist ideas regularly repeated and regurgitated in the blogs I read. They sound sensible but what are they based upon? Thought? Fact? Data? Opinion?

As ideas are communicated, the content and context changes. This form of “peer review” is in many ways beneficial, but facts can be distorted through misreadings or assumptions.

As a quantitative researcher I deal with hard figures and base my findings upon these. Assumptions are should be anathema to what I do. However research methodologies are far from flawless and one of the more problematic attributes is that it represents behaviour or attitudes at a particular point in time.

This is only of partial benefit to strategists and planners, who need to be able to predict trends. This therefore requires a level of speculation and anticipation, but they need to be based on something. But what?

I read the blogs of many successful planners and strategists, and so they must be doing something right. But to what extent should the prevailing themes of the blogosphere be treated as gospel?

In fact, this is more of uncertainty over “social media experts” than strategists per se, but why spoil a good title…

The problem with speculation

Image credit: http://www.flickr.com/photos/lolita8fotos/

This post/rant was prompted by Pew Research publishing their Future of the Internet study.

“Responding to an invitation to participate in an online survey, 578 leading internet activists, builders and commentators submitted their ideas about the impacts networked technologies may have on world societies by 2020, with an additional 618 stakeholders also participating, for a total of about 1,196 participants sharing their views”

In fairness, I haven’t read the study – it was the principle behind it that got me thinking. What were these predictions based on? As experts expressly invited, you’d imagine that their speculation would be based on some form of study and research, but to what extent should all evidence be footnoted to let us objectively assess their predictions?

Now while they are ultimately speculating, speculation from some people is better than that from others. As Mark Cahill said

“I see Social Media experts who are continually spouting the mantras of the lexicon, yet when you look at their “about me” pages (if they even have their own sites) you find they’ve never built anything.”

Despite some bloggers being well known outside of the internet, a blogger’s identity can be carefully constructed to emphasise certain areas and mask others.

On the Internet nobody knows you're a dog

The distortion effect

As ideas spread, there is the danger of turning peer review into an echo chamber. Mark Cahill also says:

“Are you really expanding the body of knowledge, or merely parroting the Internet A-listers who catch your fancy?”

What was a snappy soundbite can quickly become a buzzword and then a cliché. How many times has the word “conversation” been used in vain since The Cluetrain Manifesto was published ten years ago? How many times did we hear the word “tribes” before Seth Godin’s book came out, and how many times have we heard it since?

Social media “rockstars” aren’t all characterised by a limited vocabulary, but it is a noticeable trend.

Just because I repeat a quotation – and possibly append it to a pretty picture – does it make it correct? Is there meaning or truth behind it? What is the original context, and how does it stand up to intellectual scrutiny?

A quote can be found to back up any argument, no matter how tenuous. How do we know that the selected quote doesn’t contradict the prevailing wisdom? Chomsky does it all the time, so why can’t we?

While this exchange on the We Are Social blog takes quotes from fiction rather than from a thesis or paper, it is nevertheless a great illustration of what I am attempting to relate:

Media CzarTo quote Inigo Montoya, “You keep using that word. I do not think it means what you think it means.”

Faris Yakob“When I use a word,” Humpty Dumpty said in rather a scornful tone, “it means just what I choose it to mean – neither more nor less.”

A house of cards

Image credit: http://www.flickr.com/photos/gibbons/

Scientific knowledge extends through building theories on top of other theories. In astrophysics, very few of these can actually be proved. So if a new discovery is made that contradicts the prevailing wisdom, then the body of knowledge has to effectively regress to the point of proof.

Is our speculation in danger of being built on shaky foundations? While there is little danger of a paradigm shift that mirrors the current financial crisis, there are often missteps. Speculation frantically repeated becomes hype.

Is Twitter the new Facebook or the new Second Life? Who knows? What I do know is the amazing resilience of newspapers to continue to shape the cultural agenda. Internet society is still very much on the fringes. The various trend sites on the web point to interesting but ultimately isolated examples of new behaviour. How many times have they actually been prescient in predicting new behaviour crossing over to the mainstream? Nassim Nicholas Taleb is now regarded as a genius because he anticipated the financial crisis – should all the financial strategists and risk assessors that missed the problem retire in shame?

Snappy quotes and pretty pictures should support the evidence. They shouldn’t be the evidence.
Themes must be built on accumulation of evidence and observation. A quote or a picture may be a better way to convey the thought, but that doesn’t discount the use of appendices or footnotes to support an argument.

Speculative insight

Strategists produce (or, at least, should produce) insights. Or as Rory Sutherland more accurately puts it, ideas.

“What is the difference between an idea and an insight? My own first stab at a definition would be this. That an insight is a sudden and potentially valuable revelation concerning what is; an idea is a sudden and potentially valuable revelation concerning what might be.”

Ideas should be supported by insights, and insights should be supported by evidence. But are they?

Richard Huntingdon believes that insights come from four areas – within, “real people not respondents”, academics and “weird shit”. Academics and a surfeit of real people can produce documented evidence borne out by real behaviour, but the other two? No matter how well informed or intelligent the person might be (and my impression of Richard Huntingdon is that he is both), they still suggest speculation.

And as Nassim Nicholas Taleb has pointed out, we are pretty useless at predicting the future. Among the reasons listed in this fascinating blog post include

  • “Hindsight bias: Past events look more logical, causal and correlated than they really are… It was almost entirely good luck.
  • Vulnerability to “noise”: We are such pattern-seekers that we look for sense in the firehose of information, and find it when it isn’t there
  • The induction problem: The past always seems deterministic. We expect the future to be a continuation of the recent past… So we constantly overreact to the “latest” news (quarterly earnings reports).
  • The survivorship bias: We tend to see only the winners/survivors and lose track of the large numbers of losers.
  • The anchoring bias: Our perception of where we are is biased by where we have come from. If we have a million dollars after living our lives poor, we think we are rich. But if we once had two million, we feel poor, a failure.

Because of these biases, we are, he argues, very poor at assessing risks. I would say we are equally poor at assessing opportunities because of these same biases.”

Where’s the research?

Is research the answer? A mixture of quantitative analysis and qualitative ideas that can be extrapolated to predict the future?

If we believe in the Donnie Darko theory of time travel and path dependent lives then sure.

But I don’t.

Going back to what Rory Sutherland says

“Most conventional research, by coating everything in a patina of rationality, completely fails to uncover the baser (and hence deeper) motivations behind human behaviour.”

Crowdsourcing opinions of future behaviour won’t produce accurate results. We are good at understanding our current behaviour, but not so good at understanding our current motivations for that behaviour. So how on earth can we know what our future motivations will be?

The synthesis

Image credit: http://www.flickr.com/photos/panamaxel/

We therefore need to combine both strategic thoughts and research. For all the evidence we may accumulate, it still needs that eureka moment to anticipate how it relates to the future. These ideas are still not guaranteed success, but it may get us closer to the truth than pure guesswork or pure facts.

As Jeremy Bullmore wrote in his absolutely brilliant “Why is a Good Insight Like a Refrigerator?”

“Conventional market research, professionally conducted, can paint an invaluable picture of the immediate past; but companies also need help in forging their futures… To generate hypotheses you need to speculate: you need to progress from the known to the unknown. But you cannot paint the future in the colours of the past… The origins of an insight are usually to be found in numbers. That’s how we know an insight to be more than airy whim; that’s how we know it has substance; that it can be tested and replicated.”

We may not be able to make predictions with 100% accuracy, but we can evaluate our past predictions. An idea or insight needs to be both provable and measurable. To stand up to scrutiny, it needs to be able to be pulled apart and assessed. The alchemy of inspiration and perspiration.

Conclusion

Planning should be based in research. But equally research has plenty to learn from the strategy side. Data that is only accurate at a single point in time dates quickly – researchers can extend the life of their work by searching for these ideas that move our industries forward.

That’s my opinion anyway. But what’s my opinion based on?

Conjecture. In this post I have made several arguments with carefully selected illustrations and quotes. Where is the data and the proof that my opinion is valid?

Is this just how blogs work? Are blogs just arenas to speculate and should I not read too much into the content? After all, would any client sign off a campaign idea based solely on what someone read in a blog?

Or perhaps I have just missed my calling? This blog is packed full of febrile speculation – what sort of researcher am I?

sk

Links – 22nd February 2009

Some of the things I’ve read over the past week and would recommend:

  • A thought-provoking article in the Atlantic on the future of TV. It argues that TV’s USP is immediacy. While there are still cultural reference points via TV, scripted shows will increasingly see TV as just another distribution pattern. TV will therefore move to concentrate on news, current affairs, live reality shows and sport. This makes sense to me given my research – TV excels at events which are essentially DTR-proof, and the most popular shows online are dramas and comedies that can be viewed at leisure and shared/discussed asynchronously. However, I would argue that successful scripted shows still need TV as that anchor point for mainstream cultural crossover.
  • Ana Andjelic has a great post on our general failure to accurately predict the future. Not only does she argue that a lot of campaigns will fail, but also that our limited perspective means we will often follow the same patterns (potentially of failure)

sk

Reblog this post [with Zemanta]

Links – 31st January 2009

As Des’ree once bemoaned, “Life, oh life, oh life, oh life”. A hectic few weeks are *fingers crossed* finally over. Rather than just watching the evening news and eating toast in that time, I also managed to read and bookmark some interesting things posted on the internet. Here is part 1 of a two-part collection of said stuff.

Insights

I’ve enjoyed the back and forth discussion regarding the nature and usefulness of insights. Richard Huntingdon used Simon Law’s presentation as a basis to provocatively state that insights do not come from the research department but from a combination of within (presumably not from those within the research department though), real life, academia and “weird shit”. This inspired several other posts.

Rory Sutherland sought to distinguish between an idea – creative – and insight – deduction.

Kevin McLean, a Qual researcher, felt that a little humility could have been used in the argument.

Will Humphrey offered a balanced summary, arguing that research findings should promote creativity and lateral thinking, but that planners should be more “ballsy” when pushing for decent research.

In terms of being ballsy and challenging, Dave Trott offers an inspiring story of changing a slender brief to a truly impactful one after going away to research the product

My own thoughts? Bad research, and bad researchers, exist. As do bad planners, creatives, account managers, product directors and so on. A project is more likely to succeed if each stakeholder is capable of implementing the necessary vision. This requires dialogue on each side – utilising specific skills and expertise to challenge, mould, amend and hone a brief. It is a researcher’s duty to provide a bespoke solution that will provide real, accurate, tangible outputs. If they don’t then they have failed. But other people in the chain have just as big an opportunity to succeed or fail.

Marketing and advertising

Rory Sutherland wonder if we can outsource media planning to the public through recommendation mechanisms within social media.

John Willshire followed up on that post with the notion that this removes control on how the message is propagated. He gives a great example of how such a scheme can easily be commoditised.

Ad Week looks at the rising relevance of shopper marketing in times of media fragmentation.

Faris Yakob uses a brilliant (fan-made) Thundercats trailer to illustrate the power and benefits of recombinant marketing

Iain Tait has a minor rant about the trend of using the themes of connection and collaboration within TV advertising

Graeme Wood remarks on ways in which the internet and social media can be used to deepen involvement in a television show

A NY Times article on ways in which the internet is being used to promote new novels

“Trust Me” – a new shot set at an advertising agency has launched in the US (NY Times). Within the show, real life brands and campaigns are placed. Personally, I think this is a great way to involve brands into entertainment in an organic way. However, in the UK product placement is currently illegal and so I wonder whether the show could ever be shown over here. Precedents are mixed e.g. we may get James Bond films with the “kerching” moments uncut but the Coca Cola drinks within American Idol are pixellated.

Rohit Bhargava on how advertisers can use consumers to help promote them

Ad Rants takes a look at Bob Garfield’s overview of the widget economy. I’m now locked out of the original article, so if anyone has access I would appreciate it if a copy could be sent my way 🙂

Claire Beale on Walker’s campaign to crowdsource a new flavour of crisp

Online video

Jim Louderback makes an excellent point in that, online, the third dimension of depth – or engagement – is far more important than reach and frequency

Mark Cuban believes online video is overhyped because the technology isn’t stable enough for mass simultaneous viewing. I would argue that this is what TV is for; online video is not TV and its benefits are different, but complementary.

And to highlight that difference, because the web is much more about discovery and experimentation, we see a huge drop off in viewing between episode 1 and episode 2 of a web series. Newteevee has a great overview of a recent report

Music

A Freakytrigger post shows that the number of new entries in the UK charts has dropped off a cliff in recent years. A negative effect of the long tail?

A very interesting Music Think Tank post on a pull music paradigm shift. There is some dissension in the comments but I found it fascinating

Tomorrow’s update will feature articles on social media, technology and the internet, and business and ideas

sk

Reblog this post [with Zemanta]

Classic Blog Posts #3

Faris Yakob on Transmedia Planning.

The crux of this notion is that brands can capitalise on our converging culture and media consumption by introducing non-linear brand narratives. In other words, different media can host different elements of a campaign. Each element works on an individual level, but by combining them can give a synergistic  (SIDENOTE: I had initially favoured synergetic but spellcheck and Google overruled) result that is greater than the sum of its parts.

It is an absolutely brilliant thought. Faris expanded on it in his essay for the IPA Diploma, which deservedly won the President’s Prize.

That is by no means a slight on the other essays submitted. Reading through the other two essays published (by Graeme Douglas and Isabel Butcher), as well as the synopses of the remaining papers, shows a depth of interesting propositions.

For additional reading, Transmedia planning was cleverly expanded upon by Jason Oke, who suggested incorporating social relationships for that extra level of interest. Incidentally, Jason has since left Leo Burnett and can now be found blogging here.

Each of these links is incredibly thought-provoking, and they come with my highest recommendation

sk