MRS Mobile Insights Conference

The first Mobile Insights Conference, hosted by the Market Research Society, took place in London this week. I was in attendance.

I’ve mentioned my thoughts on the word insight several times in the past. Did I take away many insights from the day? No. Were the talks useful? In parts. Was it worth attending? Yes

Although the first use of a mobile phone for research was over ten years ago, the techniques available on the devices are still in their infancy. This means we are still in the experimentation stage of understanding what can be achieved on mobile – compelling innovations and applications of research do exist, but are rare.

As such, the projects discussed over the course of the day were largely (but not exclusively) small and tactical. It would have been great to hear of any companies really embracing mobile research to inform strategy, but perhaps we haven’t yet got to this stage.

Given the relative lack of application, the day largely focused upon the mechanics of mobile research. A topic not particularly attractive to research buyers – I counted two research buyers on the delegate list alongside attendees from research agencies and suppliers (discounting speakers). While it important to collaborate as an industry, the realities of competitive advantage meant that, with the odd exception, research agencies aren’t going to be particularly candid about learnings when they are speaking to other research agencies.

Those gripes aside, I did take some several useful things from the day.

Tim Snaith, OnePoint Mobile Surveys – Defining the Future

After some opening remarks from this OnePoint colleague Neil Jessop, Tim Snaith took the floor to talk about the immediate future. Some of the points he made include:

• “If we define mobile research in a certain way, it will be adopted in a certain way”. Instead, we should be open-minded as how people ultimately choose to engage with the mobile will define how we can engage with them
• There is currently no knowledge base in mobile so we need to build foundations with case studies and open reports
• Downtime on the computer is decreasing because there is so much on there to keep us busy. Mobile is now the device for downtime, creating opportunities for surveys
• Blending computer and mobile research isn’t ideal, since the space restrictions on mobile mean you are effectively asking different questions
• International mobile research needs to consider the different regulatory and technological requirements in each market (for instance, different networks use different character sets in SMS)

Alex Wilde, Globalpark – Tracking trends

Alex defined the four types of research that can be conducted on a mobile as a survey (e.g. invited by a banner ad), diary, storytelling (capturing stimulus) and netnography.

An issue with surveys is that there are over 3,000 combinations of mobile handset, browser, operating system and so on. Ensuring usability is therefore a challenge. Within survey design, he advocated drop down boxes rather than radio buttons, and limiting text boxes to a single line field

With regards to some of their mobile studies

• They receive 35% of responses within the first hour (Lightspeed say they received 60% of completes in the first 15 minutes and 90% within the first hour – evidently research design will be more indicative than overall methodology trends)
• When survey respondents were asked after a few questions if they would like to continue the survey and, if so, where; 70% of iPhone users said they would continue on their phone but 90% of non iPhone users said they would prefer to continue on their computer
• 90% of respondents were willing to share their location via GPS

Tom Webber, Nielsen – Smartphone trends and opportunities

Tom shared several statistics with the audience:

• Smartphone penetration in the UK grew from 12% to 20% in Q1 2010 (smartphone defined by an operating system such as iOS or Android)
• As the prices of phones rises, long term ARPU falls (presumably this was down to unlimited data charges and the need to bundle SMS rather than charge individually?)
• Net Promoter Scores for smartphone owners are fifteen points higher than those for featurephone owners (26 vs 11)
• 64% of iPhone users use apps daily, compared to 52% of Android users and 38% of other phone users
• The Apple App Store has 84% satisfaction; the Android Marketplace has 81%. Apple performs better on range of apps and download experience; Android wins on ease of browsing and discovery
• American 13-17 year olds sent an average of 3142 texts a month in Q1 2010 (I believe this figure includes twitter and facebook updates); the figure for under 12s was 1152
• In the US, Twitter accounts for 42% of phone messages sent; Facebook for 16% (though I believe GSMA data has Facebook far ahead in the UK, at least in terms of time spent)

Patrick Hourihan, Yahoo! – APPetite

Patrick (Disclosure: an ex Essential employee although, like in Ghostbusters, our streams never crossed) was the first speaker to give a presentation, rather than read a report, and it was probably my favourite talk of the day in terms of interesting content (if not practical application).

After some general industry statistics (such as 11m social network users via mobile, 5.7m people downloading games and 11m using mobile media for entertainment, he ran through the methodology for APPetite

• Online diary and forum
• Focus groups in London and Manchester
• Depth interviews with in situ mobile usage
• 2,000 person online survey among 16-65 year old mobile media users (done online as mobile doesn’t give the depth or the same level of representativeness)
• Mobile app survey using Research Now’s real-time data collection tool (I think the questions were regarding which app was used, where and how it compared to online)

Things the research covered either qualitatively or quantitatively include

• Consumer expectations have shifted from functional use to emotional use
• Nearly as many use mobile for entertainment (58%, if you include social networking) as they do functional (63%)
• There is a claimed decline in snacking, with longer usage sessions
• Brands have been mapped to times of day, with the BBC, Facebook, Google and Yahoo! in use across the entire day
• Apps have the wow factor among users, and satisfaction of services (such as Facebook) is higher for the app than the for the web (NB: This wasn’t broken down by operating system)
• However, there is some confusion over what an app is (e.g. it could be a  shortcut to a website)
• 55% don’t have a preference between an app and web so long as they can fulfil their specific needs
• When a mobile service doesn’t work, 44% blame the brand, 34% the phone and 29% the network

Siamack Salari, Everydaylives – Using mobile for ethnography

Siamack’s talk was the only brazen sales pitch of the day, but it was justified since he was the only person speaking with a genuinely groundbreaking (or so it seemed to me) research technique – a mobile application where people can capture audio, video and pictures, tag it and upload it to a central server to be sorted, filtered, analysed and edited.They can comment and feedback on the information others compile.

The iPhone app cost £5,000 to develop (whereas the BlackBerry app cost around £60,000) and will allow ethnography to be conducted on a larger scale. Previously, he would film 2 hours of footage a day over several days, conduct interviews separately and then spend time editing the reams of footage.

The app come with a small price, but he is shortly going to launch a consumer version, which is free and would allow people to effectively sell their lives to willing buyers. He mentioned that, independent of this, Coca Cola and P&G approached him about fitting out their workforces with the application.

Learnings from his first four studies are

• Not all iPhone users are familiar with iTunes, and so will need to be walked through app installation
• A template of how to shoot video is required for a consistent quality of submissions
• Research needs to be designed carefully to avoid it altering people’s behaviour
• He has facilitated post-filming opt-outs, where he gives people an email address and a code, in case they want to have their privacy protected in their friends’ film

Leonie Hodge, Channel 4 and Anthony Cox, Sparkler – Using SMS to uncover drivers of viewing

As a cost effective alternative to ethnography (and allowing a greater sample), Channel 4 commissioned Sparkler to conduct research into drivers of viewing using 24 depth interviews, 9 in-home evening visits and 35 media diaries lasting for 6 days using a combination of mobile and paper.

Essentially, on days 1, 3 and 5 Sparkler would text the respondents to ask them what they were going to watch on TV that evening. They would send several messages (manually) if requiring clarification or reminding. On days 2, 4 and 6 people would fill in a standard consumption diary of what they watched the previous evening.

The research showed that there were very few appointment to view programmes (a disappointment?), and the majority of consumption was last-minute or spontaneous decision making.

Sparkler’s advise was to

• Get the stakeholders to pilot the technique, in order to engage them with the process
• Keep it as simple as possible for the respondent
• Design the outputs so analysis can be as efficient as possible

AJ Johnson, Ipsos MORI – Using mobile to gain greater customer insights

AJ was the exception in that he was quite open and candid about Ipsos MORI’s various experiments with mobile research – both successes and failures. Initiatives include

• Using mobile for passive media consumption for radio (via ambient sound recording) and posters (via GPS)
• Quick polls for PR purposes
• Comparisons to online research, which showed that mobile surveys took longer with shorter answers and a lower response rate, but that quality of response (recall, in this instance) was higher
• Multimedia diaries – they invited 1,000 people to take part in recording their weekend activities. 200 agreed to take part, but only 37 provided usable material. They had asked for 4 diaries a day from people, but received an average of 2.7 per person (they tended to be photos)
• Software they have been experimenting with includes Microsoft Pivot and The Link (unsurprisingly, I can’t find the link)

Gavin Sugden, T-Mobile – Measuring customer satisfaction via SMS

Gavin was also exceptional, in the sense that his was the only example of SMS research being used more strategically. T-Mobile changed their customer satisfaction survey from a 20 minute CATI survey to an SMS survey of around 6 questions – 3 standard questions and 3 from a battery of 15. They found SMS would be a better capture method and more cost effective than CATI, IVR or the mobile web, though Gavin didn’t seem to have a problem with privacy, saying that people could delete the messages if they didn’t want to respond (T-Mobile customers would be contacted within 24 hours of purchase in-store, web activity or phoning customer services).

With people only eligible to complete one survey in 60 days, they are seeing a 20% response rate.

They have an online reporting tool, which allows analysis of quantitative data and aut0-coding of verbatims.

They currently use the results for performance management (e.g. benchmarking stores) but are moving towards coaching and development of staff, and seeking of ways to improve customer satisfaction. They are also going to start include campaign awareness questions

Liam Corcoran, Fly Research – Insights into alcohol consumption

Liam walked us through a case study from a study he ran for Mintel, looking at alcohol consumption of young adults. Rather than hazy recollections, they used mobile research for in situ response. At 10pm each night, they would send an SMS with a link to a 7 question mobile internet survey.

He also talked a bit more generally about their mobile panel research.

• They tend to receive 60% response rates from their panels and 20% from cold sample
• 50% of their responses come within the hour and the vast majority within 24 hours
• He wouldn’t recommend a mobile internet survey of more than 15 questions
• Mobile is a great tool for a follow-up to online research, if just a couple of additional answers are needed

Sarah Sanderson and Jason Vir, Kantar Media – Integrating mobile with other research techniques

Sarah and Jason gave the last talk by going through a very recent project they ran during England’s World Cup campaign (presumably the project was ran specifically for the Conference; it must be nice to have the resources to be able to do that!)

The research was designed to access the emotional response in real-time, which they felt would be more honest and truthful

If I remember correctly, they:

• Recruited people to the project based on answers to questions in TGI Sport+ and TGI Postscript (the TGI omnibus)
• Got some people to take part in an online community to discuss the tournament
• Asked these people to submit videos and pictures of them during/after the game (I think 7 people did this, I’m not sure if the community was larger)
• Sent a WAP survey at half time to a larger sample. 207 responded, 90% of these during the half time interval

Neil Jessop closed the day with a few remarks, including a prediction that there would be more mobile surveys than online surveys. This was the one contentious comment of the day (in my opinion), and this sums up my overall feelings of the event. There were a few interesting asides and thoughts, but there was nothing at the event that really made me stop and think. A shame, but perhaps I was expecting too much.

sk

Image credit: Research-Live

Advertisement

Research 2008: The Great Debate (Part 4 of 4)

Go to part 3 here

Part 4 contains (1) Web 2.0: Capitalising on communities, (2) Closing remarks and (3) My conclusions

Day 2 Session 3: Web2.0: Capitalising on Communities

The final formal session of the conference was also the most fun. There was little particularly relevant to my work, but it is a subject I am interested in and the passion of the speakers was obvious.

Mario Menti of GMI and Diana Derval of Derval Research opened the session with a look at research within Second Life.

Research 2008: The Great Debate (Part 3 of 4)

Go to part 2 here

Part 3 contains (1) The Big Planning Debate: Is research failing in the boardroom and (2) Guaranteeing a return on investment

Day 2 Session 1: The Big Planning Debate. Is research failing in the boardroom?

And so onto Day 2. To open the session, Vanella Jackson of Hall & Partners introduced a video containing some very interesting quotes from business leaders. These included wanting intelligence and not insight, wanting a solution rather than the research, research is too often used as insurance rather than as a forensic analysis and that research is the only tool in the marketing mix that hasn’t substantially changed in the last 12 years. A call to arms then.

Rupert Howell of ITV gave a very entertaining keynote speech before the debate/Q&A began.

Research 2008: The Great Debate (Part 2 of 4)

Go to part 1 here

Part 2 contains (1) Web 2.0: Harnessing the Potential for Business, (2) Honing Business Skills and (3) Pecha Kucha… And that’s why I love market research

Day 1 Session 3: Web 2.0: Harnessing the Potential for Business

This session kicked things off after lunch. It was chaired by Richard Young, who was the most enthusiastic and involved of all the chairs I saw over the two days. While this could have become overbearing, he generally let the speakers talk for themselves.

Dan O’Donoghue from Publicis gave this session’s keynote.

Research 2008: The Great Debate (Part 1 of 4)

Part 1 contains (1) Introduction, (2) Welcome to the Great Debate and (3) Ensuring Transformation

Introduction

The 2008 Market Research Society conference was the third year I have attended, and the one I took the most away from. Entitled Research 2008: The Great Debate, the event’s stated aim was to “change business through better customer understanding”, which seemed to involve a lot of soul-searching about how business leaders perceive the industry.

This review became a lot longer than I initially envisaged, and so I have split it into four sections – morning and afternoon sessions of both days. Apologies in advance for either including too much detail or for misquoting or misattributing any information (my handwriting isn’t the most reliable of things)

Links – 13/03/08

I’ve been feeling under the weather recently. So while I have some ideas for blog posts, I haven’t actually written them yet. Sunday should be the time to rectify that.

I’m going to the MRS Conference on both days next week; please say hello if you will be there too.

Blog related links

• Trendy business names (Smithsonian)
• Conversation by design (David Armano slides)
• Attempted Wikipedia cover-ups (wisebread)
• 50 most powerful blogs (Observer)
• Why Indie bands sell out (Advertising Age)
• The battle for Wikipedia’s soul (Economist)
• Form + Function for digital design (Ad Week)
• Individuality within corporate blogs (Groundswell)
• Massify aims to create films bottom-up (NYTimes) NB This will be tricky
• Controversial Wikipedia pages (AV Club)
• Hulu’s ad format (Advertising Age) NB I am annoyed this is US only

Random links

• 100 NES games converted to Flash (not as good as I remembered)
• 13 days in hell (shoot some zombies)
• Difficulties with the English language (kottke)
• Rise of the Machines (Daily Mail fearmongering)
• Hair metal hall of fame (Hamptonroads)
• Nixon’s nuclear chicken (Wired)
• Conspiracies that happened (Cracked)
• Best Zombies of all time (OMGHorror)
• Alan Moore primer (AV Club) NB Alan Moore will be namechecked in an upcoming blog
• A Choose Your Own Adventure mapped out (nice visualisation)
• How 280 programmes are tied together (from St Elsewhere to The Wire via Dr Who)

Of these, my top 3 would be Why Indie bands sell out, Individuality within corporate blogs, and Alan Moore primer, with 100 NES games converted to Flash thrown in for a long lunchbreak.

sk

Thinkbox Event – TV & The Brain: How Creativity Wins

Last Wednesday, I attended the Thinkbox event TV & The Brain: How Creativity Wins. The half-day conference explored how psychology plays a role in brand communications and advertising. The argument is that we should be looking towards the emotional and not the rational.

As a researcher, this is a challenge. Rational messages are easy to measure – emotions aren’t. I went into the event wanting to build up my knowledge on the theory, to learn of any practical applications and to leave with ideas on how to improve our understanding of advertising evaluation.

The event was split in two – half on theory, half on application. Personally, I found the first half far more rewarding. My knowledge of psychology was limited to Malcom Gladwell books, but the three excellent speakers broadened my horizons considerably and left me with a lot of things to ponder. I found the second half a disappointment. There were few specifics and the talks were dangerously close to sales pitches.

Tess Alps, Chief Executive of Thinkbox, opened the event in the customary fashion of selling TV as a medium.

Trust me, I’m a researcher

Everyone knows the aphorism lies, damn lies and statistics. We continue to use and rely on them but do we – and more important, do our audience – trust them?

The government isn’t so sure, and has some stats that suggest that trust is lacking. Leaving aside the contradiction of trusting statistics that indicate a lack of trust in statistics…
– In 2005 ONS reported that only 17% of people believe that official figures are produced without political interference and only 14% say the government uses official figures honestly
– A Eurobarometer report put the UK last of all the 27 EU member states in terms of public trust in official statistics

This is why The Statistics Commission (obviously shortened to Statscom) is being replaced by the UK Statistics Authority (headed up by no less a person than the President of my former college Sir Michael Scholar). The two main issues for this change, which do amount to more than a rebranding, are:
There was no limit to the time the government could sit on, and thus spin, figures
There were no powers to constrain the spin put on figures, or ability to compel the government to report objectively or accurately

It is interesting that the government is worried that a lack of trust is symptomatic of overt spin, for this is a given in the type of research I work in. Paraphrasing an old joke, with 4 researchers you can get 9 opinions. As evidence, note the preening and posturing that accompanies each ratings report – whether it is RAJAR, NRS, ABC or another measurement tool. Each party is able to find a positive message from the results. It is no surprise when competitors offer completely contradictory accounts of the same figures.

Research is regularly commissioned with a hypothesis that the client wishes to prove, and results can be fashioned to suit that hypothesis. Numbers will be open to interpretation and it is difficult to get a truly objective opinion. Unfortunately, exaggeration of data to suit an agenda is not uncommon. Contentious research will provoke arguments from both sides. The more exaggeration gives more scope for sniping, and reduces faith in the findings.

The Market Research Society can control the quality of the research process by bestowing and removing accreditation, but it has no powers of compulsion over how research is reported. And nor should it. An independent commission to verify the reporting of research would be unworkable – for cost, time, size and client confidentiality issues among many others.

However, more effective self-regulation can build trust in data reports. Data will always be open to interpretation, but integrity is vital. What good is the spinning of a good message when the truth is distorted? Ultimately, inaccurate recommendations will only come back to hurt the client.

This brings me on to some of the recommendations laid out in what is presumably the final report by Statscom – Official Statistics: Values and Trust – recommendations that all would be wise to take on board.

1. It would do more for trust if there were greater public engagement by the professionals, regardless of the political ripples that might then create

Absolutely. Opening research to debate can evolve the conversation. Issues can be dissected and probed further. Not everyone will agree but identifying issues of agreement and issues of contention can help clarify the message.

2. Better explanation of the messages contained in official statistics is likely to be one of the most potent ways to ensure they are better used – and thus deliver greater value. Better explanation means widespread and routine dissemination of statistical commentary written in a way that is understandable to a broad readership, with key messages highlighted and the limitations of the statistics considered in the context of their likely use.

Research designs aren’t infallible. Acknowledging the restrictions of the design and the level of statistical confidence in the figures underlines openness. Messages conveyed may then be caveated, but there is less room for suspicion.

3. Users also need to be confident that the statistical products have not been amended (or concealed or delayed) so as to suit a particular policy or argument. These two components – quality and probity – are central to the concept of being trustworthy.

MRS Accreditation should ensure quality within the research process. Without an ombudsman probity is difficult to relate, but openness should help build trust.

4. The key to quality in the statistical output is that the producer must advise the potential user on the merits of the estimates and the potential pitfalls of relying on them.

Definitely. Understandably, research agencies don’t want to damage client relations by picking holes in a hypothesis. Indeed, in a competitive pitch, the drawbacks of the research methodology may be glossed over. But when research is being used to influence business decisions, the client needs to be fully informed of the margins for error in the research.

There will always be agendas and there will always be individual interpretations in research. By opening up debate to both the methodologies and recommendations, we can build on the conversation, and that can build on trust.

sk