Notes from MRG Conference 2011

A couple of weeks ago I took part in a short session at the 2011 Media Research Group Conference, which took place in London. I took some notes during the day (mainly with the earlier speakers). They are below and in chronological order, though firstly a quick exec summary:

The four papers I enjoyed most were

Synthesising these talks, my key take-aways were:

  • Run lots of prototypes and versions
  • Ask audiences what they think, rather than just infer from behaviour
  • Set up the tests in such a way to drive people towards the behaviours/answers you desire
  • Be aware of contextual reasons that might provide counter-intuitive answers

And now for the detail…

 

Tim Harford – Problem Solving In a Complex World

Tim Harford, author of books such as The Undercover Economist) , initially walked through examples of problem solving such as

  • Archie Cochrane – a Prisoner of War who conducted experiments to find out what was making people ill in the camp
  • Thomas Thwaites – a student who took 9 months and spent over £1,000 to try and make a toaster from scratch and even when cheating largely failed
  • Cesar Hildago – who has mapped 5,000 product categories. But Wal-Mart has 100,000 types of product in a store, and in New York there are probably 10bn

His point was around the God Complex – the conviction that no matter how complex something is or how little data is available, you know the answer. It is dangerous and yet you see it everywhere.

We need to step away from the god complex as we can’t solve things in one step. Instead, we gradually learn over time through trial and error.

For instance, Unilever wanted to create a new nozzle through for their detergent production. They hired a mathematician who failed to sufficiently improve it. Instead, they created ten random computer generated models and picked the best. They then created ten variations of this. They repeated this process twenty times. Ultimately the nozzle was much improved, although they don’t know why.

Business successes are random processes – there is no silver bullet for the perfect CEO or strategy. However, instilling a start-up culture allows experimentation to see what is best. Google has a target failure rate of 80%, but this failure has to be quick, rather than being too big to fail. In order to do this, we have to overcome loss aversion.

In the BBC documentary about Fermat’s last theorem,  Goro Shimura said in reference to his colleague Yutaka Taniyama :

Taniyama was not a very careful person as a mathematician. He made a lot of mistakes, but he made mistakes in a good direction so eventually he got the right answers. I tried to imitate him but I found out that it is very difficult to make good mistakes.

Tim fielded a couple of questions relating to popular business books

  • Tom Peters’ In Search of Excellence profiled many companies to see what made them successful, but three years after the book was published around one third of them were in trouble (e.g. Wang, Atari). Were they actually excellent, or is excellence fleeting?
  • James Surowiecki’s Wisdom of Crowds is often misunderstood as he himself said that it only works in specific situations – when expert judgement is no help and where the crowd can be polled independently (Duncan Watts has shown how randomness becomes important when things are dependent

Claire McAlpine – Mediacom – How are you integrating behavioural economic thinking into your work?

Inspired by thinkers such as Steven Johnson (Where Good Ideas Come From) and Chip & Dan Heath in addition to Thaler & Sunstein etc.

Hunches are where we collide ideas – these could be our ideas over time, or our ideas with other people’s. For instance, the Gutenberg printing press was inspired by the wine press.

We need to overcome cognitive biases (such as picking the second cheapest wine on the list) and recognise things such as information deficit and availability bias. We are more Homer Simpson than Spock – we are not rational agents. We may have good intentions but these can quickly be forgotten if we are in a “hot state”.

There are three stages to integrating behavioural economics

  • Identifying the behavioural context
  • Identifying the behavioural journey
  • Identifying choice context and ultimately creating choice architecture

Claire gave the example of Special Constable recruitment. By identifying two choice contexts – career and inspiration – Mediacom were able to frame their media strategy (both in terms of creative and placement) for two separate audiences

By understanding how behaviours differ, we can seek out how to encourage the desirable ones to be replicated. The ultimate goal is to be able to switch the default behaviour, which we often resort to as a mental shortcut.

 

Mark Barber (RAB) and Jamie Allsopp (Sparkler) – Media & the Mood of the Nation

Mark and Jamie went through the research findings of this research which covered 3,500 smartphone survey responses from 1,000 people, qualitative depth interviews and diaries and EEG brain scan experiments.

The research came about from the general move in advertising from systematic (logical) to heuristic (emotional) processing, and observations that advertising works better in mood-enhancing environments.

The findings were framed using James Russell’s Circumplex Model of Affect, which places results on two -5 to +5 scales of arousal (energy) and valence (happiness).

Radio was compared to both TV and online. While all displayed rises in happiness and energy, radio showed the highest average increases in total and across the most dayparts. While this may be caused by other activities people are doing while they listen to the radio, it nevertheless means that people are in a more receptive frame of mind when it comes to processing advertising messages.

 

Becky McQuade (Sky) and Anne Mollen (Cranfield School of Management) – Online Engagment: We might be getting there

Anne said that there are two schools of thought with engagement

  • It is bankrupt as it is not a metric since it is too abstract and not credible (unlike retention and acquisition)
  • It is viable (she is in this camp)

The academic studies in this area have been focused on perceived interactivity and telepresence (her paper is here), but it hasn’t as yet properly been joined up to commercial requirements.

Her definition of engagement is “cognitive and affective commitment to an active relationship” and requires

  • Utility/relevance
  • Pleasure/enjoyment
  • Dynamic and sustained cognitive process

Using Survey Interactive, they ran an online pop-up survey with 60 engagement statements (reduced from an original list of 150) on 12 point scales across 14 Sky websites (and on a NetMums panel), resulting in over 12,000 responses. This found four drivers of correlation. From the largest to smallest, these are:

  • Cognitive processing e.g. enjoyment
  • Temporal needs e.g. hedonic and utilitarian value (what we need and want)
  • Self-congruence (identity with the brand)
  • Social identity (context, environment, peer to peer communication)

Conversely, engagement isn’t

  • A measure of human behaviour – there was low correlation between engagement and time spent, frequency and recency
  • Behavioural footprints (actions such as subscriptions or likes) – there was only a small positive correlation among a subset of those engaged
  • Activism (such as loyalty) – engagement is context dependent and not a behavioural type

The study was specific to advertising, and found those engaged had higher ad recall, improved core message delivery, more favourable opinions towards the brand and a higher likelihood to purchase (but not higher purchase intent).

Becky and Anne closed by saying for engagement to be viable it has to have a close relation to ROI and KPIs. Their NetMums study showed engagement has an impact on trust, satisfaction, loyalty and add responsiveness and has a high positive correlation with the Net Promoter Score.

Anne isn’t linked exclusively to Sky and will talk to others on a confidential basis around her engagement scale, but given academic competition to publish there is only a limited amount she can say publicly.

 

Stuart McDonald (News International) and Euan Mackay (Kantar Media) – Show Me the Money: Proving the value of tablets

Given that the results of the research are being used to inform News International’s commercial strategy, they didn’t really go into how value was proved. The research was conducted among News International’s subscriber base, and tested interactive advertising on a beta app (The Times app doesn’t yet have advertising) against a premium engagement index, comprising of perceptions of an ad being

  • Memorable
  • Relevant
  • Engaging
  • Trustworthy
  • Premium

 

Richard Curling (Google) – YouTube Skippable Pre-Rolls: Measuring the power of choice

Given “Hurry sickness” – the malaise where people feel short of time so perform tasks faster and get flustered by any delays – we’re increasingly looking for shortcuts.

YouTube “true view” means that users get to choose their adverts – if they don’t like an advert, they can skip it. Advertisers only pay for adverts that are viewed all the way through. Google interpret a high view rate as a high quality score, and this will factor in alongside price when bidding in an auction for advertising space. Thus, high quality ads are rewarded (though arguably very low quality advertising can benefit from a lot of free, interrupted views).

Using Ipsos MediaCT, Google tested the effectiveness of these ads using biometrics (heart rate, respiratory rate, skin conductance, motion- via Innerscope), depth interviews and eye-tracking. These found that both skipped and “true view” ads scores higher on their engagement metrics, though the true view ads scored highest. However, this wasn’t as clear-cut as you might expect – people opting in might have higher expectations and so could be harder to please. Conversely, the engagement of people forced to watch an ad might pick up towards the end as they get ready for their content to start

Richard’s recommendations for advertisers were to

  • Entertain the user, since you are the content
  • Be clear, and support user choice
  • Embrace “natural” targeting

 

Afternoon sessions

I was paying less attention to these, since I was mentally rehearsing my speech

  • Ross Williams and Becky from Ipsos MediaCT presented their “Big Brother Research – Who’s Watching Who?”, which combined social media monitoring of Big Brother properties. with Facebook polls. While Big Brother wasn’t as big as other properties, it had a 80-20 proportion of comments to likes on Facebook (indicating an engaged audiences), while alternative programmes had the opposite ratio
  • Steve Cox of JC Decaux presented “Airport Live” – following a small number of passengers at both their departure and arrival airports to see what they were noticing
  • Matthew Dodds of Nielsen and Nick Metcalfe of the Telegraph presented “Telegraph Print + Net Online Multiplier study” which took 5 groups of people (Telegraph print readers, Telegraph online readers, readers of both, non-print readers with matched demographics, non online readers with matched demographics) from UKOM to test uplift in advertising measures
  • The Good The Bad & The Ugly of Media Research was hosted by Max Willey and featured myself, Dave Brennan, David Fletcher, John Fryer, Stef Hrycyszyn and Loraine Cordery talking about whatever we wanted to for three minutes. David Fletcher won the prize, for his tale of why people think they want online dashboards but don’t

 

Industry Updates

  • BARB is looking into a non-linear database that would report on archive programmes on demand, and catch-up from longer than seven days after transmission. They will also evaluate, and possibly publish topline results of, the TV+online data
  • POSTAR – now have tube and bus data, and are looking at GPS devices to see how people move around. This is being validated and they hope to get it into a reporting system soon
  • NRS – concentrating on fusion with UKOM data, but hope to get more granular data and move online in future
  • RAJAR – moving the diary online, and continuing to explore the viability of passive meters
  • IPA – bedding down touchpoints. Touchpoints 3 included word of mouth, mobile internet, social media, gaming and on-demand. Touchpoints 4 will bring in tablets and apps, and change from a device-first structure to a content-first structure. It now has 60 subscribers (including each of the top 20 agencies) and has launched in the US. They are also piloting an app to go alongside the diaries
  • UKOM – the past year has been about stabilisation after some data issues. The contract is currently out to tender and whomever is successful (they would take over in January 2013) would look to measure all devices and locations (ie beyond home/work fixed internet to include mobile and video)

sk

Nassim Nicholas Taleb and Antifragility

At the Royal Society on Thursday I saw Nassim Nicholas Taleb (Distinguished Professor of Risk Engineering at Polytechnic Institute of New York University) in conversation with Rohan Silva (Senior Policy Advisor to the Prime Minister) and Matthew Taylor (Chief Executive of the RSA).

The subject was Taleb’s latest book – Antifragility – which he hopes isn’t as simplified as his previous book (he seemed to dislike the fact that The Black Swan was bought and read in airports by people who subsequently misinterpreted it). I think he has succeeded, as at times I struggled to grasp his arguments.

Nevertheless, I’ll have a go at summarising it. The images below are taken from his slides, available alongside the audio of the event, at the RSA website.

I believe Antifragility draws on Prospect Theory (incidentally, Michael Lewis has recently written about Daniel Kahneman – one of the main proponents of the theory – in Vanity Fair). The thesis is that organic entities are fragile, and that potential harm is non-linear to the size of the event (see below). In fragile systems, potential losses outweigh gains.

Antifragility is the opposite of fragility – where gains outweigh losses (robust is something different). The parallel Taleb drew was:

  • Fragile – Sword of Damocles (risk of fall)
  • Anti-fragile – Heidegger Hydra (cut one head off and get two back)
  • Robust – Phoenix (stays stable)

To give an example, it is more harmful to jump once from 100m than 100 times from 1m.

Taleb-nonlinearharm

This means that the unpredictability of Black Swan effects costs more when the size of the event is large.

Black Swan event

This means (I think) that risk is concave, and there is more pain than gain in an event. For instance, traffic is concave – it is fine to the point of over-leverage, when the cost of travel will suddenly massively increase.

Antifragile events are convex. Taleb believes that bottom-up structures are convex, while top-down dictated structures are concave. We cannot effectively design systems as we cannot fully understand them, and so there will always be more harm than good.

The result of this theory is that large events, large organisations and large governments should be avoided. “Too big to fail” is catastrophic, and Taleb is firmly in favour of smaller organisations – on a moral level (it smaller societies you are close to the result of your decisions and so may feel shame) and an economic level (the cost of liquidating one large bank is far more than the cost of liquidating ten banks one tenth of the size of the large bank).

Uncertainty makes mistakes costly, and thus both businesses and governments should remain small. Taleb suggested that if Tesco suddenly ran into difficulties then the government would have to bail out a supermarket. He feels the government should only intervene in things that can’t organise organically, and is thus advising the government on how to make its institutions smaller (such as splitting the NHS into localised, autonomous units).

Taleb also talked a bit about risk management, and said that the models were flawed as they don’t combine risk and growth –  to get rich you must survive. He feels slow and steady growth is preferable to high growth that cuts corners, since that will ultimately crash. He pointed out that perfect growth is Madoff, and that Greenspan and Brown’s attempts to end boom and bust have led to where we are now (he claimed that Brown was more dangerous than Blair as Brown thought he understood the world).

He also mentioned that within small systems, stresses can be positive. For instance, we fast to kill cancer calls. In exercise, we should combine slow and steady exercise (such as walking) with the occasional intense burst (such as lifting weights at the gym). Up to a point, what doesn’t kill you makes you stronger.

Taleb doesn’t see big as permanent – Google could quite easily disappear tomorrow. To get around organisations growing in size, he recommends a contract saying that if a business wants to have the option to be bailed out then they can never pay out bonuses. Since people will want bonuses, they will intentionally keep their business small in size to not require bail-outs.

He closed by saying organic organisations are perishable and that each day they exist their life expectancy drops by a day. Whereas non-perishable entities – such as ideas – increase their life expectancy each day. For instance, if an idea has existed for 25 years then it will exist for 25 more.

Ultimately, the talk was very thought-provoking but I’m not sure whether I agree with it or not. On the one hand, I can see the diseconomies of scale of large organisations, but on the other hand businesses will have become large through success. That success might have been random and unwarranted, but they nevertheless survived and so must have done a few things correct. Taleb appears to advocate limiting them in order to protect against the possibility of them failing and causing problems for the economy. This risk aversion and diversification may be prudent, but I’m not sure how fragile this system really is (particularly if it made up of bottom-up organisations).

sk

My MRG Conference 2011 speech

At the MRG Conference 2011 (pdf link to programme here) I was given a three-minute slot to talk about anything I wanted under the banner “Six industry speakers share the good, the bad and the ugly from our industry”.

This is (roughly) what I said:

Good afternoon everyone. As Research Manager for Mobile, Social and Syndication at the BBC I’m understandably enthusiastic about these areas. So today I’m going to take the first area I mentioned – mobile – and explain how its characteristics make it appropriate as a research platform.

The first is universality – mobile has more coverage than any other research method. A big claim maybe, but Ofcom stats say that

  •  77% of households have PC-based internet
  • 85% of adults have a landline
  •  91% of adults have a mobile, and this rises to 98% among 16-54 year olds

More than 91% of the UK might have a home and can be reached by door to door, but realistically, once you factor in accessibility and interviewer safety, mobile will have the largest potential audience for research – though the key word there is potential; there is still the small hurdle of getting the audience’s contact details.
The second characteristic I want to mention is relating to proximity. More than any other platform, mobile is our go-to device. It is nearly always turned on, it is nearly always on our person and thus it is when we have some free time or are bored it is the first thing we turn – in fact I can see a few phones in the audience now. This captive audience on mobile has massive potential for research purposes, though we need to ensure what we ask them to do is both interesting and relevant. Easier said than done, perhaps.

But, this is predicated on the notion that we need our respondent to interact. We can do many great things on mobile – video diaries, photos, status updates etc and in real-time. But one of the real strengths of mobile is its latency. Why ask people what media they are consuming when mobile sensors can match sound to TV and radio; record web browsing, use GPS to plot outdoor reach and time spent; and soon use near field communication to record sales of newspapers and magazines. Admittedly, not all phones can do this just yet, and privacy is obviously an issue, but again, there is big potential.

The young will drive this, for mobile is a youthful medium – 16-24s say they would miss mobile the most if they had to give up media. These behaviours might not be mainstream yet, but a dozen years ago owning a mobile wasn’t mainstream, and look where we are now. But there is also a second aspect to this point around youth, and that is that the medium is nascent. We’re still learning all the time – no one can say they’ve cracked mobile in terms of capturing and utilising. This is a huge opportunity for research agencies both big and small to move into.

This is an opportunity because it doesn’t yet exist. There is plenty of innovation at the edges, but the market isn’t yet mature. So while I’ve identified several benefits to mobile research, they come with caveats and are more theoretical than practical. So as much as I want to say mobile is good, I can’t really. I’ve talked about the universality, the go-to nature, the latency and the youthfulness. That’s U.G.L.Y and it ain’t got no alibi, it’s ugly.

sk