Research vs Planning

Image credit: http://www.flickr.com/photos/darkumber/

Preamble

I’ve learned a lot from reading and interacting with various blogs. But something has been bothering me for a while now.

What if it is all complete rubbish?

That’s blatantly trolling and patently not true – I read a lot of fascinating and thought-provoking posts. But the extent to which I should implicitly trust their veracity is something that is concerning me.

There are definitely zeitgeist ideas regularly repeated and regurgitated in the blogs I read. They sound sensible but what are they based upon? Thought? Fact? Data? Opinion?

As ideas are communicated, the content and context changes. This form of “peer review” is in many ways beneficial, but facts can be distorted through misreadings or assumptions.

As a quantitative researcher I deal with hard figures and base my findings upon these. Assumptions are should be anathema to what I do. However research methodologies are far from flawless and one of the more problematic attributes is that it represents behaviour or attitudes at a particular point in time.

This is only of partial benefit to strategists and planners, who need to be able to predict trends. This therefore requires a level of speculation and anticipation, but they need to be based on something. But what?

I read the blogs of many successful planners and strategists, and so they must be doing something right. But to what extent should the prevailing themes of the blogosphere be treated as gospel?

In fact, this is more of uncertainty over “social media experts” than strategists per se, but why spoil a good title…

The problem with speculation

Image credit: http://www.flickr.com/photos/lolita8fotos/

This post/rant was prompted by Pew Research publishing their Future of the Internet study.

“Responding to an invitation to participate in an online survey, 578 leading internet activists, builders and commentators submitted their ideas about the impacts networked technologies may have on world societies by 2020, with an additional 618 stakeholders also participating, for a total of about 1,196 participants sharing their views”

In fairness, I haven’t read the study – it was the principle behind it that got me thinking. What were these predictions based on? As experts expressly invited, you’d imagine that their speculation would be based on some form of study and research, but to what extent should all evidence be footnoted to let us objectively assess their predictions?

Now while they are ultimately speculating, speculation from some people is better than that from others. As Mark Cahill said

“I see Social Media experts who are continually spouting the mantras of the lexicon, yet when you look at their “about me” pages (if they even have their own sites) you find they’ve never built anything.”

Despite some bloggers being well known outside of the internet, a blogger’s identity can be carefully constructed to emphasise certain areas and mask others.

The distortion effect

As ideas spread, there is the danger of turning peer review into an echo chamber. Mark Cahill also says:

“Are you really expanding the body of knowledge, or merely parroting the Internet A-listers who catch your fancy?”

What was a snappy soundbite can quickly become a buzzword and then a cliché. How many times has the word “conversation” been used in vain since The Cluetrain Manifesto was published ten years ago? How many times did we hear the word “tribes” before Seth Godin’s book came out, and how many times have we heard it since?

Social media “rockstars” aren’t all characterised by a limited vocabulary, but it is a noticeable trend.

Just because I repeat a quotation – and possibly append it to a pretty picture – does it make it correct? Is there meaning or truth behind it? What is the original context, and how does it stand up to intellectual scrutiny?

A quote can be found to back up any argument, no matter how tenuous. How do we know that the selected quote doesn’t contradict the prevailing wisdom? Chomsky does it all the time, so why can’t we?

While this exchange on the We Are Social blog takes quotes from fiction rather than from a thesis or paper, it is nevertheless a great illustration of what I am attempting to relate:

Media Czar – To quote Inigo Montoya, “You keep using that word. I do not think it means what you think it means.”

Faris Yakob – “When I use a word,” Humpty Dumpty said in rather a scornful tone, “it means just what I choose it to mean – neither more nor less.”

A house of cards

Image credit: http://www.flickr.com/photos/gibbons/

Scientific knowledge extends through building theories on top of other theories. In astrophysics, very few of these can actually be proved. So if a new discovery is made that contradicts the prevailing wisdom, then the body of knowledge has to effectively regress to the point of proof.

Is our speculation in danger of being built on shaky foundations? While there is little danger of a paradigm shift that mirrors the current financial crisis, there are often missteps. Speculation frantically repeated becomes hype.

Is Twitter the new Facebook or the new Second Life? Who knows? What I do know is the amazing resilience of newspapers to continue to shape the cultural agenda. Internet society is still very much on the fringes. The various trend sites on the web point to interesting but ultimately isolated examples of new behaviour. How many times have they actually been prescient in predicting new behaviour crossing over to the mainstream? Nassim Nicholas Taleb is now regarded as a genius because he anticipated the financial crisis – should all the financial strategists and risk assessors that missed the problem retire in shame?

Snappy quotes and pretty pictures should support the evidence. They shouldn’t be the evidence.
Themes must be built on accumulation of evidence and observation. A quote or a picture may be a better way to convey the thought, but that doesn’t discount the use of appendices or footnotes to support an argument.

Speculative insight

Strategists produce (or, at least, should produce) insights. Or as Rory Sutherland more accurately puts it, ideas.

“What is the difference between an idea and an insight? My own first stab at a definition would be this. That an insight is a sudden and potentially valuable revelation concerning what is; an idea is a sudden and potentially valuable revelation concerning what might be.”

Ideas should be supported by insights, and insights should be supported by evidence. But are they?

Richard Huntingdon believes that insights come from four areas – within, “real people not respondents”, academics and “weird shit”. Academics and a surfeit of real people can produce documented evidence borne out by real behaviour, but the other two? No matter how well informed or intelligent the person might be (and my impression of Richard Huntingdon is that he is both), they still suggest speculation.

And as Nassim Nicholas Taleb has pointed out, we are pretty useless at predicting the future. Among the reasons listed in this fascinating blog post include

• “Hindsight bias: Past events look more logical, causal and correlated than they really are… It was almost entirely good luck.
• Vulnerability to “noise”: We are such pattern-seekers that we look for sense in the firehose of information, and find it when it isn’t there
• The induction problem: The past always seems deterministic. We expect the future to be a continuation of the recent past… So we constantly overreact to the “latest” news (quarterly earnings reports).
• The survivorship bias: We tend to see only the winners/survivors and lose track of the large numbers of losers.
• The anchoring bias: Our perception of where we are is biased by where we have come from. If we have a million dollars after living our lives poor, we think we are rich. But if we once had two million, we feel poor, a failure.

Because of these biases, we are, he argues, very poor at assessing risks. I would say we are equally poor at assessing opportunities because of these same biases.”

Where’s the research?

Is research the answer? A mixture of quantitative analysis and qualitative ideas that can be extrapolated to predict the future?

If we believe in the Donnie Darko theory of time travel and path dependent lives then sure.

But I don’t.

Going back to what Rory Sutherland says

“Most conventional research, by coating everything in a patina of rationality, completely fails to uncover the baser (and hence deeper) motivations behind human behaviour.”

Crowdsourcing opinions of future behaviour won’t produce accurate results. We are good at understanding our current behaviour, but not so good at understanding our current motivations for that behaviour. So how on earth can we know what our future motivations will be?

The synthesis

Image credit: http://www.flickr.com/photos/panamaxel/

We therefore need to combine both strategic thoughts and research. For all the evidence we may accumulate, it still needs that eureka moment to anticipate how it relates to the future. These ideas are still not guaranteed success, but it may get us closer to the truth than pure guesswork or pure facts.

As Jeremy Bullmore wrote in his absolutely brilliant “Why is a Good Insight Like a Refrigerator?”

“Conventional market research, professionally conducted, can paint an invaluable picture of the immediate past; but companies also need help in forging their futures… To generate hypotheses you need to speculate: you need to progress from the known to the unknown. But you cannot paint the future in the colours of the past… The origins of an insight are usually to be found in numbers. That’s how we know an insight to be more than airy whim; that’s how we know it has substance; that it can be tested and replicated.”

We may not be able to make predictions with 100% accuracy, but we can evaluate our past predictions. An idea or insight needs to be both provable and measurable. To stand up to scrutiny, it needs to be able to be pulled apart and assessed. The alchemy of inspiration and perspiration.

Conclusion

Planning should be based in research. But equally research has plenty to learn from the strategy side. Data that is only accurate at a single point in time dates quickly – researchers can extend the life of their work by searching for these ideas that move our industries forward.

That’s my opinion anyway. But what’s my opinion based on?

Conjecture. In this post I have made several arguments with carefully selected illustrations and quotes. Where is the data and the proof that my opinion is valid?

Is this just how blogs work? Are blogs just arenas to speculate and should I not read too much into the content? After all, would any client sign off a campaign idea based solely on what someone read in a blog?

Or perhaps I have just missed my calling? This blog is packed full of febrile speculation – what sort of researcher am I?

sk