Recommended reading – 30th April 2010

This week, I am mostly recommending:

Will Humphrey on the differences between PR and advertising, having now worked in planning departments for both sides

Bud Caddell presents a very thorough overview of the Theory of Planned Behaviour. Not a bad basis for questionnaire design in certain instances.

Sam Page has an excellent analysis of Jeff Francouer’s swing which shows how powerful statistical evidence and observational evidence can be when properly combined. I promise I’m not planning to post a Baseball link every week, but this is a really strong piece of work, and should be readable for all irrespective of sporting preferences.

Tom Slee’s eloquent rebuttal to Clay Shirky’s Collapse of Complex Business Models. I like Shirky as a writer, and I don’t mind the occasional extrapolation of anecdotes if they prompt further discourse and discussion, but some excellent points are raised.

Ken Auletta has a long, thought-provoking piece in the New Yorker on how Amazon versus Apple are lining up in the battle of the book business.

sk

Advertisements

Avoiding insights

I really don’t like using the word “insight”.

As I wrote here, the word is hideously overused. Rather than being reserved for hidden or complex knowledge, it is used to describe any observation, analysis or piece of intelligence.

And so I’ve avoided using it as much as possible. In an earlier tweet, I referred to the Mobile Insights Conference that I’ve booked to attend as the MRS Mobile thing. And I even apologised for my colleague (well, technically, employer) littering our Brandheld mobile internet presentation with the word.

But this is irrational. I shouldn’t avoid it, if it is the correct word to use. After all, substituting it for words like understanding, knowledge or evidence might be correct in some instances, but not all.

Does it really matter? After all, isn’t a word just a word? As someone once said, “What’s in a name? That which we call a rose by any other name would smell as sweet“.

But he’s talking complete rubbish. Because words do matter. They cloud our perceptions. It is why brands, and brand names, are so important. And why blind taste tests give different results to those that are open.

In fact, this emotional bond we have with words has undoubtedly contributed to my disdain. And this should stop. So I vow to start reusing the word insight, when it is appropriate.

But when is it appropriate? I’ve already said that an insight is hidden and complex, but then so is Thomas Pynchon and he is not an insight.

In the book Creating Market Insight by Drs Brian Smith and Paul Raspin, an insight is described as a form of knowledge. Knowledge itself is distinct from information and data

• Data is something that has no meaning
• Information is data with meaning and description, and gives data its context
• Knowledge is organised and structured, and draws upon multiple pieces of information

In some respects it is similar to the DIKW model that Neil Perkin recently talked about, with insight replacing wisdom.

However, in this model – which was created in reference to marketing strategy – an insight is a form of knowledge that conforms to the VRIO framework.

• Valuable – it informs or  enables actions that are valued. It is in relation to change rather than maintenance
• Rare – it is not shared, or cannot be used, by competitors
• Inimitable – where knowledge cannot be copied profitably within one planning cycle
• Organisationally aligned – it can be acted upon within a reasonable amount of change

This form of knowledge operates across three dimensions. It can be

• Narrow or broad
• Continuous or discontinuous
• Transient or lasting

How often do these factors apply to supposed insights? Are these amazing discoveries really rare and inimitable, and can they really create value with minimal need for change? Perhaps, but often not.

And Insight departments are either amazingly talented at uncovering these unique pieces of wisdom, or they are overselling their function somewhat.

When I’m analysing a piece of privately commissioned work, a finding could be considered rare and possibly inimitable (though it could be easily discovered independently, since we don’t use black box “magic formula” methodologies). But while it is hopefully interesting, it won’t always be valuable and actionable.

But if it is, I shall call it an insight.

sk

Image credit: http://www.flickr.com/photos/sea-turtle/2556613938/

My name is my name

So says Marlo Stanfield. And he has a point.

Reputation means a lot. But reputation is about perception, and there are multiple perspectives in which it can be viewed.

Broadly, reputation can be thought of in four inter-related spheres

• Yourself – your personal brand
• Your organisation (this itself can have several facets, if your organisation is part of a larger conglomerate or affiliation)
• Your industry
• The wider public

Marlo is concerned with his personal reputation among people in the industry – “the game”. He isn’t so worried about the other facets.

With the prominence of polling in the upcoming general election, the research industry is contemplating its reputation among the wider public.

I don’t think it really matters.

This election is more partisan and contentious than any I recall (most likely driven by the likelihood of change, rising prominence of online media giving a voice to more people, and the novelty of the leadership debates). Pot-shots, such as those against YouGov, are inevitable. This article from Research Live shows how YouGov aren’t doing themselves any favours in their need for speed (and this is leaving aside their associations with The Sun/Murdoch/Conservative Party).

I don’t think it matters because the research industry is rarely public facing – the only publicity it really receives is through political polls and PR research.

I’ve written about the problems with PR research in the past, but there is evidently a market for it and so the method prospers. It might damage the reputation of the industry to the wider public but outside of recruitment  (of staff and respondents/participants) it isn’t really relevant.

As Marlo noted, it is industry reputation – for yourself and your organisation – that really matters.

It is similar to the advertising industry. Successful companies have a lot of brand equity through the quality and associations of their work – Wieden & Kennedy and Nike, Fallon and Cadbury, HHCL and Tango, and Crispin Porter & Bogusky and Burger King, to give but four examples.

But what proportion of the general public has heard of these companies, let alone recognises and appreciates their work? Not many. Is it a damning indictment of the strength of the marketing industry that it fails in promoting the most basic thing – itself? Not really. Companies attract talent and business through their successes and image – public perception doesn’t factor.

Ray Poynter is rightly concerned with the the ethics of market research but for me, the importance of this is in maintaining business links. There is no adequate means of policing the research industry – anyone can knock on a door and say they are doing a survey – so it is not a battle worth fighting.

Companies stand and fall by the quality of their work – or at least the perception of it within the industry. Sub-standard work that is openly criticised will only harm long-term prosperity.

Self-regulation and recognition, whether through a recognised body like the Market Research Society, or at a more ad hoc level, can achieve this through highlighting good and bad practice.The research industry needs to be more vocal in showcasing good work, and castigating poor work.

This in turn will filter to the individual level, where the talented and ambitious will compete to work for the top companies. This in turn strengthens the work, and thus the industry. It could even permeate to the public.

There is no quick fix to improve the standing of an industry, and in some cases it isn’t necessarily desirable. Rather than look to the big picture, we should focus on the more immediate challenges.

If we all concentrate on undertaking the best possible work, then a strong reputation – for ourselves, our organisation and our industry – will follow.

sk

NB: The clip of the scene with the quote is below (it is from Series 5, so beware of potential spoilers)

Recommended Reading – 24th April 2010

I decided against posting a list last week, and instead held out until I had a decent number of quality links worth sharing. I now have a high-quality list, which can be found below:

• Laura Miller on Salon suggests how Apple could learn from Amazon in its use of metadata, if it is serious about selling books. I argued last week that metadata is a valued, and overlooked, resource that offers a lot of potential for analysis – particularly on mobile.

• James McQuivey on the Forrester blog argues why Hulu should be available for subscription. The comparison to Netflix is a good one, but whether the networks (not to mention the operators that carry them) would ever approve this is a completely different matter.

• Asi Sharabi lists 8 sins of nu-marketing folk. Sample quote for the sin of dogmatism: It’s easier to shout “it’s all about this!” (’this’ being the buzz-word of the day: engagement, relationships, co-creation) than to scrutinise the context face the uncertainty, and admit the complexity

• Dirk Singer highlights 20 free social media evaluation tools.Though Fresh Networks might argue that the quality of these tools is too low to bother with (at this time).

• Joel Rubinson discusses “second order decision strategies” and how behavioural economics can help us understand, and thus improve, the pre-planning stage of the purchase journey.

• Alastair Gordon writes a very interesting piece on what the ownership of market research companies could look like in 2020. He posits that operations suppliers could takeover some of the client-facing project management/analysis companies. I’m not sure I agree – research quality is largely hidden and thus undervalued, so I don’t think these companies would have the ability to successfully integrate in this way – but a thought-provoking theory.

sk

Criteria for agency selection

According to my CIM coursebook, the following criteria should be used to shortlist agencies

1. Area of expertise
2. Quality of existing clients
3. Reputation of principals and experience of staff
4. Agency fees and methods of charging/payment
5. In-house resources
6. Geographical cover

While the selection of the agency should be based upon

1. Credentials – track record and feedback
2. Creative techniques – evidence of creativity and innovation
3. Staff – number, tenure and experience
4. The agency – resources, objectives, service level agreements
5. Specialism – focus
6. Price – clear and reasonable structure
7. Legal – methods to ensure compliance with regulations
8. Pitch – whether it met the requirements of the brief

The list isn’t fully comprehensive, but it acts as a reasonable guide – assuming you want to ignore slightly shadier aspects like favouritism

It can act as a useful checklist when pitching for new business. Of course, this only considers absolute performance/measures. When in a competitive pitch, the relative strengths become most important as pitching agencies are traded off against one another.

With open pitch processes, comparative advantage can be identified and relative strengths can be focused upon. With closed/opaque bids, this isn’t possible. So an agency will need to estimate where its relative and absolute strengths lie.

A good agency will have the relevant market intelligence to make a decent stab at this. A bad agency won’t. (Though of course industry fragmentation and lack of market definiton makes the potential competition so broad that it may not be possible/efficient to undertake)

Incidentally, the book also lists five key roles for an account planner (derived from Yeshin).

1. Defining the task and bringing together the key information
2. Preparing the creative brief
3. Creative development, including being the “custodian” of brand values
4. Presenting to the clients to convey concepts and defent rationale
5. Tracking performance

The focus of this can be adjusted so that it is also applicable to researchers – on the assumption that planners/researchers play a central role throughout the project or campaign. Some people might disagree about that.

sk

Image credit: http://www.flickr.com/photos/mistersnappy/2282846520/

The mobile phone is the drill to extract the data

Last week I wrote a blog post entitled “If data is the new oil, we need a bigger drill“, where I complained that we weren’t making enough use of the potential data available to us.

That post was in relation to online research. But on reflection, the opportunity is far greater elsewhere.

On the mobile phone.

Introduction

And, as far as I am aware, it is an area even more underexploited than online data capture. Aside from the odd application (such as Everyday Lives – which looked very similar to Evernote last time I looked), mobile survey panels (such as One Point) or academic experiment (Contextphone in Helsinki), I’m not aware of any innovations in mobile.

Which is a shame, since it is arguably the most powerful media platform for data capture. The Wikipedia page on the 7th mass media lists the eight unique benefits mobile has. Of most relevance are that the mobile is

• Always on
• Always (well, usually) carried on the person
• Available at the point of creative inspiration
• Highly personal, and personalised

The unique benefits of mobile make it an ideal instrument for both active and passive data capture – for explicit answers and for implicit inferences.

Forms of data capture

I’ve drawn an arrow below of the five primary means a mobile can capture information. It is very much an early draft, so feedback or criticism is very much appreciated.

Background capture

As mobile technology advances, devices incorporate more features that produce information on the location of the phone, and thus the user. These include:

• Time – the time itself, and the time it takes to do something via a clock and stopwatch
• Date – via a calendar
• Space – via GPS
• Proximity – to people, objects or events via GPS, bluetooth or RFID chips
• Movement – in three dimensions via an accelerometer, or inferred through GPS and clock
• Environmental factors – through thermometers, altitude readers and so forth

In addition to location, the following can also be determined through past or current behaviour:

• Spend – via the in-built payment mechanism
• Social graph – via the address book
• History – via cookies or memory
• Broad character traits – by how the phone has been customised or used

While in future, these will be augmented with innovations such as voice and face recognition (a Google Goggles type of service).

Either on their own or in combination, these features facilitate some extremely powerful data capture. They effectively allow us to understand the “where” and “when”, and potentially the “with”.

But it is only the first level of information capture.

Activity Capture / Activity Follow-up / Prompted Activity

I’ve grouped these three stages together, as they are essentially variations on a theme.

The mobile phone has a large number of features and services that can be used for data capture. These include

• Voice calls
• Text messaging
• Voice recorder
• Note taker
• Calendar
• Bluetooth
• Games
• Camera/scanner
• Video camera/editor
• Music player
• Web browser
• Email/social network use
• Application downloads/use
• Shopping/purchasing

The most passive form of data capture is in recording the functions that a person uses their phone for. Forms of analysis this facilitates include

• Combining activities with the data dimensions outlined in the first section for understanding of individual uses
• Utilising path analysis across all feature uses to understand how the phone is used as a single device, rather than as a collection of services
• Converting phone calls to text, and then using sentiment analysis to infer meaning across all forms of communication.

These aspects augment the “where”, “when” and “with” with the “what” and “how” – at least in terms of mobile phone behaviour.

A slightly more active form of data capture would move closer to capturing the “why”.

For instance, a push notification could be triggered when a certain activity is undertaken. This could request a simple answer to a question.

For instance, if I were to use my camera to take a picture, I would know

• Where it was taken
• When it was taken
• What it was taken with
• How it was taken (landscape or portrait, flash or natural, first attempt or fifth)
• Potentially who/what it was taken of
• Potentially who the person was with at the time

But it wouldn’t be known why the photo was taken, or whether the person was happy with the photo taken. A simple question or two would solve that.

An even more active version of data capture would be to explicitly ask the person to use their phone for a particular person. For instance, they could be asked to use the camera to scan each item they buy on the high street or to use the voice recorder or note taker each time they spot a certain advertising campaign. These methods are used by a couple of organisations – MESH spring to mind – but have little noticeable traction to date.

This manual mechanism may eventually be superseded, as technology allows us to automate more of the data capture. Its only real relevance would be in forcing someone to participate in a behaviour where they naturally wouldn’t.

Direct questioning

As should be obvious, the more explicit forms of data capture are those that are most prevalent – primarily because their implementation is independent of technological advancement. For instance, we’ve always been able to interview people over the phone. As technology improves, the interfaces underpinning this method will also improve – we will move from SMS surveys to java to html to html5 or native applications, with touch screen drag and drop functionality.

Benefits

As I mentioned in my previous post, we aren’t close to reaching the level of data capture that is possible. We need to augment explicit questioning with the context that can be inferred from the situational data collected. The mobile phone, moving across space and time and with its unique benefits, offers even more scope for collecting meaningful data.

Potential uses for the data capture include

• Calculating sleep quality/efficiency (an iPhone app already does this, to a degree)
• Monitoring movement, speed and proximity of people across an environment could be used for town planning
• Alternatively, it could be used to plot the efficiency of layouts in supermarkets. If the phone could calculate eye line (it would probably need to be attached to a necklace), it could even inform how the shelves are stacked
• Providing an understanding of people’s lives – when using and not using their phones.
• Exploring how things spread across mobile phones. For instance, one person could undertake a type of behaviour, come into contact with someone else, and then the second person undertakes the behaviour. Network effects could be used to identify the mythical influencers
• Tracking spend can be used for financial management
• A networked calendar/diary could become predictive e.g. rescheduling a meeting to take place 15 minutes later due to traffic
• Tracking movement can improve the measurement of exposure to outdoor advertising
• Sound recognition could be used for radio or TV exposure, and improve out-of-home consumption measurements
• Inefficiencies of usage could be explored e.g. the time it takes to connect a phone call can be compared across devices and networks

Practical obstacles

Evidently, the previous section was quite speculative and fantastical, but I hope it underlines the potential. Nevertheless, several obstacles need to be overcome before this point is reached

• What is the best way to collect such information? Within the operating system? The network/SIM? Via the web or an application? The O/S with control of the API would appear to be best placed, but do they have the inclination?
• Although phones are always on and regularly used, they are also regularly upgraded. Information collected would need to be portable for long-term tracking
• Similarly, a phone is more susceptible to breakage, theft or loss
• Background data capture would be a tremendous drain on the battery
• Effective data capture would require an entire network of people using it – this is highly unlikely, not least because there will always be a significant proportion of people for whom a mobile phone will just be a device to make and receive emergency calls
• More behaviour will be transferred to the mobile, but it will only ever capture a small proportion of our lives
• Coverage and connectivity isn’t good enough (in the UK) for full capture – unless information can be stored natively before it is uploaded to a central server
• Massive issues of data protection and privacy. Some people (such as Nicholas Felton) would enjoy tracking their movements, but I suspect – outside of paid-for testing – few would appreciate it. Particularly since the mobile is the most personal of devices Imagine if large corporations were able to track the movements and social graph of its employees through mobile phone usage?

Conclusion

This post seems to have been sidetracked into future gazing, but my underlying point remains. The technology is available for us to capture far more information – and thus understanding – then we currently do. Organisations should look to harness and utilise this data, to provide contextual meaning to what people are doing.

Thoughts on how we could do this – or on how people are already doing this – would be much appreciated

sk

Image credits: http://www.flickr.com/photos/kioan/3011984637/ and http://www.flickr.com/photos/_parrish_/2575256484/

http://en.wikipedia.org/wiki/Seven_mass_media#Eight_unique_benefits_of_mobile_as_a_media_channel

If data is the new oil, we need a bigger drill

“Data is the new oil” is a phrase I’ve heard a few times recently, notably from Gerd Leonhard.

And it has the potential to be true. Data can be extremely powerful and extremely valuable (though I’m a researcher, so I would say that).

Yet few companies seem to be exploiting this to its full potential. The research industry is particularly poor in making the most of available data.

Google is an obvious example of a company on the right track, and there are isolated examples of companies doing interesting things – OK Cupid’s blog for example. Or advanced Sabermetrics, that looks to account for every possible variable that can influence a play.

Every possible piece of information around a decision, an action or an opinion should be sought to be captured. Only then can the context be understood.

There might be issues with analysis paralysis. But Moore’s Law is consistently providing the processing power that minimises this problem.

If data is the new oil, we need a bigger drill

At some point, you would have seen a presentation using the picture to the right.

It is a horrible cliché.

But clichés tend to propagate because there is some truth to their message.

And whether icebergs or oil fields, the analogy holds for data.

Most information we hold is taken at face value – what people say.

But formal research with questions and answers is only part of the solution. We don’t live in a vacuum where an individual reply is sacrosanct. There is our social, herd mentality. There is our willingness to be nudged. Randomness. Our subconscious. Our poor recall. Our aversion to admitting certain things. And, of course, our inability to answer certain questions.

I can’t remember who tweeted it but I really liked the comment:

“The problem with the general public is that they have an average IQ of 100”.

Pithy, and possibly a little patronising, but true. We need to account for the biases in our ability to answer questions – whether intentional or not. And we do that by capturing the context. Moving away from just what people say.

This is the Mehrabian principle (the 7-38-55). Where people are conflicted, we should pay the least attention to what they say. Instead, we should put more weight to the non-verbal communication.

What are people doing?

How are they doing it?

Where are they doing it?

When are they doing it?

And possible even modeling why they do it.

These are the hidden depths. We need to move from the stated to the observed, and to augment the recorded with the inferred.

With research methodologies, there are three main areas that need to develop. “Formal” research companies are already losing ground to analytics in the online area, and this problem will only get worse unless action is taken.

(Non-researchers: Thank your for persevering with this post up until this point, I hope my general point has resonated with you. If not, I’d love to hear why. However, the rest of the blog post concentrates on the operations side of research, and so you may want to call it quits here)

1. Centralised, historic panel information

Surveys are horribly inefficient, since the same questions are often asked each time. Online panel providers seem reticent to assume or re-use information from their profiling studies. Reasons I have heard – and my responses – include

• Information may be out of date – some things will be but some demographic details will remain constant (like date of birth and, barring the exception to the rule, gender) and I’d be fine taking some mesofacts for granted if – for example – the data held had been submitted within the previous 12 months
• Not all panel members will have completed all the profiling questions – surely some information is better than none. Additionally, the questions could only be asked to those where the information is missing
• Information needs to be collected within the survey in order to factor into quotas or screeners – this is an artificial technological limitation and there is no good reason why this can’t be achieved

Once these limitations are overcome, there is scope to do some quite powerful things. A central databank could build powerful profiles of (claimed) consumer behaviour across multiple dimensions. Individual surveys could be coordinated to add to this dataset. For instance, I might have a question on my survey on the brand of TV the respondent owns. If I were incentivised by the panel provider in some way, I would consent to framing this question in a standardised format, so that the data collected can be stored centrally and contribute to further profile information to be reused on additional surveys. Incrementally, extremely rich datasets of panel members can be built up, so that the number of questions needed to ask directly can be reduced

This idea of a central databank also has powerful implications for qualitative recruitment. Information on people tends to be collected informally by local recruiters – a centralised, formal databank comprising answers to all previous screener questionnaires would be a much more efficient – and accurate – form of recruitment.

Clearly, there are data protection issues with this. If consent were sought, and benefits made clear (shorter, more relevant and more varied questionnaires), then it could gain traction. And while it runs contrary to current guidelines, additional consent could result in highly targeted marketing messages – mimicking for instance the advance product testing P&G use their Tremor panel for.

2. Metadata

In online surveys, some metadata is collected but its uses tend to be related to data quality. Examples include

• Average length of questionnaire
• Ensuring there is a certain gap between survey invitations
• Measuring drop-out rates
• Removing respondents that continually “straight-line” or answer in obvious patterns

Yet there are many ways in which metadata can be used for analysis – either within the survey itself or for general knowledge on segmenting the types of panel users. Examples include

• Deriving geodemographic data from IP addresses
• Measuring word count and time spent answering open-ended questions
• Using time spent making decisions (such as in conjoint surveys) to calculate “velocity of opinion”*
• Segmenting those that tend to complete surveys shortly after the initial invitation into a “fast response” panel
• Calculating preferences from the order in which modular surveys are completed (though most surveys remain linear)

*For velocity of opinion to be an effective measure, panel members would need to be segmented based on quality of response (including average time spent answering questions) so that normalised scores can be created, and deltas measured.

This has potential cross-over with cookie-tracking web use panels, while metadata can be a valuable form of meta analysis in online communities – used to both encourage participation and understand both individual responses and the community culture as a whole. Someone,  probably Tom, once wrote about the types of data that can be collected in a community to help encourage response (primarily through gaming responses), and it really resonated with me.

3. Integrated analysis tools

This is more of a personal request to make my life easier, rather than something that is currently absent or overlooked, but surely multivariate analysis tools could be built into survey scripts and automated. The current system – sending completed datasets to statisticians to run through their proprietary tools – seems incredibly inefficient. If questions are asked in a consistent way, then the analysis method would be consistent and thus could be automated.

This post seems to have turned into a general moan about a lack of innovation in data collection, but my underlying point is that only a small fraction of the data collected is currently used, and used effectively. This seems a shame, but it is also an opportunity since there is room for significant improvement in the way we currently operate.

sk

Image credit: http://www.flickr.com/photos/21734563@N04/2385731147/