Ten things I learned from the New MR Virtual Festival

My previous post included all of the notes I took while listening into the New MR Virtual Festival. This post contains the key things I took away from the day, and have subsequently been mulling over in the 2 months (!) since.

New MR Virtual Festival header

NB: All slides shown below are taken entirely without permission. If a creator objects to its use, please contact me and I’ll remove it.

1. The boundaries between participation and observation can (and, in some circumstances, should) be blurred

Although Ray Poynter touched on the range of methods in which research can overtly or covertly be conducted online, Netnography (cf. Robert Kozinets) is – to me – one of the more interesting. To be effective, it needs to have the research both participate and observe the environment of interest.

Erica Ruyle argues that observation (or lurking) is fine in the initial stage, since the norms and cultures need to be understood and respected. But active participation is vital in order to get than “insider” knowledge and to be able to read between the lines of the interactions.

This is a difficult proposition to promote commercially as a) the time investment (and thus cost) required will be huge and b) the researcher will need to have a personal as well as professional interest in the topic in order to both be accepted by the community and accept the community. For instance, how many researchers would turn their nose up at being asked to take part in World of Warcraft for 6 months?

Nevertheless, in the right circumstances it could prove to be a fascinating and rewarding exercise.

2. Convenience samples can still be worthwhile

It is true that all non-census research incorporates a convenience sample to some extent. But some methods require more convenience (and thus are less representative) than others.

Annelies Verhaeghe highlighted some of the issues to be aware of when conducting social media research – particularly that we should resolve ourselves to not always know who we are speaking to or monitoring.

Furthermore, something I had not considered but makes sense is that even though companies trumpet the volume of data they scrape and collect, only a sub-sample of that will be analysed due to the diminishing returns of going deeper into a very large data set.

If we’re able to augment social media research with other techniques or data sources – Annie Pettit mentioned some of the benefits of combining social media research with surveys – then it can be a very valuable and insightful method of getting real-world information on past actions and thoughts.

3. Respondents shouldn’t necessarily be viewed equally

Both Rijn Vogelaar and Mark Earls talked about premises realised more thoroughly in their books – The SuperPromoter and Herd respectively.

Segmenting the audience isn’t a new phenomenon – we often restrict our universes to who we are interested in – but within these universes perhaps we should pay more attention to some individuals more than others – particularly given the complex social interactions that cause ideas and opinions to spread. I’m not clever enough to be able to incorporate full network theories into any of my research – in the manner of Duncan Watts, for instance – but perhaps there is an argument for applying simple weights to some projects, to account for some opinions becoming more important than others. Or perhaps it is too contentious to implement without proper academic grounding and proof.

4. More of an effort needs to be made to meet respondents on their terms

Betty Adamou joined the likes of Stowe Boyd in saying that email is on the decline among younger people. This is problematic for online surveys, given that online panels are predominantly administered via email. Given the trends, perhaps we should be looking to Facebook, Twitter, instant messenger etc for both initial recruitment of these audiences and then allow them to dictate how we can contact them to alert them with new surveys. I’m not sure whether a note on a Facebook group could be as effective as an email, but it is certainly worth exploring.

5. Survey structures can do more to take advantage of the online environment

Our media and communications channels have fragmented but the content providers retain a centralised hub of controlling activity. Why can’t the research industry do this? John Dick talked through Civic Science’s modular research methodology, whereby questions are asked in chunks of two or three at a time, but combined at the back-end to build up extensive profiles of respondents.

This approach makes intuitive sense. In face to face research, the main cost was in finding people to speak to. Thus, once they were located, it was efficient to collect as much information as possible. The web is abundant with people, who are time-poor. The cost isn’t in finding them, it is keeping them. People could easily answer three questions a day if there was the possibility of a prize draw. They would be less willing to spend 30 minutes going through laborious and repetitive questions.

There are clearly downsides to this method and plenty of issues to overcome regarding data quality assurances, but the notion of Facebook users answering a couple of questions a day sounds like a feasible way to collect information among people who might be unwilling to sign up to an online survey

6. Surveys don’t have to be boring or static…

Another aspect of the online world that should be explored further is the level of interactivity. Many online surveys are straight ports of face to face surveys – a shame when there are so many more things that a web survey can – in theory – be capable of.

Jon Puleston of GMI highlighted several of their experiments in this area. Interestingly, although interactive surveys take longer, respondents are more engaged, enjoy them more and give “better” answers. I particularly like the idea of timing respondents to give as many answers as possible within a given timeframe. This appeals to people’s competitive nature, and means they’d spend far longer on it than they normally would.

Jon Puleston of GMI at New MR Virtual Festival

The King of Shaves case study was very interesting. Rather than a 1 minute introduction for a 10 minute survey, this example reversed the process. People were given a detailed briefing on the role of a copywriter, and asked to come up with a creative slogan. In subsequent testing, seven “user-generated” ideas scored better than the advertising agency.

7. But we should be aware of the implications of survey design on data capture…

Jon’s examples showed how framing questions can improve data collection. But Bernie Malinoff warned us that even minor superficial changes to a survey can have a big impact on how people answer questions. For instance, the placement of the marker on a slider scale can heavily impact the distribution of answers.

Bernie Malinoff at the New MR Festival

Bernie also had some important lessons in survey usability – ranging from the wordiness of the questions (something I’ve been guilty of in the past) to the placement of error messages and how they can influence subsequent responses.

Surprisingly, his research found that survey enjoyment was comparable among people who did traditional “radio button” style surveys versus richer experiences, and that people were less willing to take part in future after having completed a flash-based survey.

It acts as a sobering counter-point to Jon’s presentation, but I inferred some caveats to this research (or perhaps I am only seeing what I want to see). I suspect some of the resistance to flash might be down to the “newness” of the survey design rather than a genuine preference for radio-button style surveys. Similarly, design iterations aren’t neutral – I wouldn’t mind different results so long as I felt they were “better” (and any methods to discourage survey cheaters are welcome). Nevertheless, it an important reminder that a better designed survey is only an improvement if it makes the survey more usable and easier to understand, and I completely agree with the final point that the industry should reinforce best practices for interface design.

8. …And whether they are suitable for the audience

Tom Ewing’s talk on gaming and research covered many interesting points, but the one that stuck with me is that it isn’t for everyone. As he points out, FourSquare holds little appeal to him (unless he wanted to be Mayor of his child’s nursery). Similarly, while the number of gamers is rising, it isn’t everyone and so we cannot assume that introducing interactive, exploratory or personalised experiences will automatically make respondents enjoy our surveys more.

Particularly since games design is pretty hard – Angry Birds and Farmville may look simple, but I wouldn’t expect any research agency to devise and incorporate something as addictive to their research methodologies. The latter in particular seems to purely encourage the completion of repetitive, monotonous tasks – not something that would benefit the quality of research outputs.

9. There is plenty of scope to improve beyond the debrief

John Clay talked about ways in which researchers can improve the way that debriefs are communicated. This is an area that many (too many) researchers need to improve upon, but an even more pressing area of improvement is what occurs after the debrief.  Spencer Murrell’s presentation on insight translation covered this.

Spencer Murrell at New MR Virtual Festival

Summary slides and executive summaries are important in debriefing research, but it is important to go beyond the report/presentation into a framework that can be referenced in future. Whether it is a model that can be stuck on a wall, or a cheat sheet that can be put on a post-it note, there are many creative ways in which the core findings of a project can be transformed into an ongoing reminder. Evidently, this could easily descend into a gimmicky farce, but it is important to remember that the debrief isn’t the end of the project. In many ways, it is only the end of the beginning. The next phase – actually using that information to improve the organisation – is the most important. Any ways in which researchers can add value to this stage can only improve their standing with their clients.

10. Online conferences can work

On the whole, I think the event can be viewed as a huge success. For an affordable fee ($50), I listened to many intelligent speakers on a variety of topics, as shown by both this post and the previous post.

There was also plenty of excellent discussion around the talks and the content on Twitter, using the #NewMR hashtag. I’m usually reticent to tweet during events, but given the lack of face-to-face contact and the fact I was facing my computer at the time, Twitter worked excellently as a forum to amplify the usefulness of the content presented.

An idea is one thing, but executing it is something very different. Aside from the odd technical hitch (inevitable given the volume of speakers from across the globe), the day ran impeccably. So Ray Poynter and his board deserve huge congratulations for not only the concept, but also the organisation and output of the event. I would wholeheartedly recommend people with an interest in research investigate the New MR site and list of upcoming events.

sk

New MR Virtual Festival notes

I’m breaking my longest-to-date blogging absence (work-life parity should soon be restored) with two versions of the same post. This is the first.

They are related to the New MR global online conference that ran on 9th December 2010, featuring speakers and moderators across Europe, the Americas and Asia-Pacific. The event was created and organised by Ray Poynter – a long-standing, committed and energetic member of the international market research community – and his management board. In addition to the core event, various “fringe” events also took place. More info on them can be found at the 2010 Festival pages on the site.

This is the larger of the two posts, where I’ve reformatted all of the notes I made on the day and supplemented them with some additional thoughts (I’ve not yet caught up on the presentations I missed for reasons such as being asleep/exhausted, while some presentations weren’t as relevant to me and so I skipped them). So while this isn’t exhaustive, there will still be plenty of words to keep you occupied for a short while.

I’ll follow it up with a shorter post outlining my key takeaways from the day, and my overall thoughts on the event.

As this is the single longest post on this blog (circa 5,000 words), I’m taking the rare step of putting the bulk of it behind a cut (the size also means it is not properly proof-read). Click through to continue (unless you are reading via RSS) Continue reading

Mark Earls – From “me” to “we”

Thanks to Mat kindly donating his ticket, I was able to go and see Mark Earls give a seminar entitled From “me” to “we” at the Royal Society.

herd by mark earlsRather shamefully, I am still yet to read Herd – the book (and associated research) on which the talk was based. This is despite regularly reading the Herd blog and even having a copy in the Essential library. As I said, shameful.

Despite this, I think I was the target audience. Along with a Q&A only notable for the rather aggressive questioning of a lady accusing Mark of ignoring “the female perspective”,  the session offered a fairly gentle precis of the book’s central theory which, if I had read it, I would of course have been familiar with.

The talk

A tenet of the book is that we’re bad at changing other people’s behaviour. To highlight this, Mark recalled a few statistics from his research:

  • Only 10% of new products survive longer than 12 months
  • Only 30% of change management programmes begin to achieve their aims
  • Mergers & Acquisitions lessen shareholder value two thirds of the time
  • No government initiative has created demonstrable and sustainable change

This is particularly worrying because behavioural change comes before attitude change – our thinking comes after the fact. We (post)rationalise rather than act rationally.

Therefore, in order to change attitudes, we need to change behaviour. And to be able to do this, we need to understand who we are. Only then can we can create solutions that work.

The Herd thesis draws upon the Asian culture of believing that humans are naturally social. We are fundamentally social with only a bit of independence, not vice versa.

Although it doesn’t sound particularly controversial, this thinking does run contrary to some well established tenets of both marketing and social theory.

According to Mark, thinking is much less important in human life than it seems. He likens us and thinking to a cat in water – we can do it if we have to, but we don’t particularly like it.

This is because it is easier to follow than think. We know our judgement is fallible and so we outsource the decision by following the crowd. But while this may work in some situations – many illustrated by James Surowieki – it is also arguably a contributing factor to the financial crisis, as financial institutions copied one another without comprehending the implications.

We therefore need to design our theories and tools to accommodate this social behaviour. It is much more rewarding to understand how social norms are created and perpetuated than it is to work on the assumption of cogito ergo sum.

Some initial thoughts

While brief, the talk certainly conveyed the need for me to read the book fully. Perhaps then some of my questions regarding the theory will be answered.

In particular, I’m interested in knowing where movements originate and whether this herd behaviour can be predicted.

For all the sheep, there must be a shepherd somewhere. Are these shepherds always designated as such – the almost mythical influentials – or do we alternate between thinking and following?

Rarely are our choices as clear cut as choosing whether to join the corner of the party where people are talking rather than the one where people are sitting in silence. Instead we have multiple choices and herds – how do we choose?

Is it a level of proximity? In the Battle of Britpop, Northerners sided with Oasis and Southerners with Blur? However, I’m from the Midlands, so was my choice one of the rare occurrences of rational choice (which would make a rather unconvincing deus ex machina) or is it purely random?

If random, then the work of Duncan Watts becomes pertinent. His modelling has suggested that in situations where groups vote up and down their favourite songs, there is no objective winner. Different simulations create different patterns. Purely random.

This creates difficulties for researchers as we like our statistical certainty. We like to have a set answer that we can post-hoc explain given the evidence. Duncan Watts’ research would suggest that research tools that build in mass opinion – such as crowdsourced tagging or wikis – are effectively meaningless. Rather than ultimately deviate towards a “correct” answer, they simply reflect the random order of participation and interaction.

Can mass behaviour be effectively incorporated into a research programme? I’ll report back with some thoughts once I’ve read the book

sk

We’re bad at changing other people’s behaviour

Only 10% of new products survive longer than 12 months

30% of change management programmes begin to achieve their aims

Mergers & Acquisitions lessen shareholder value 2/3 of time (pwc)

No government initiative has created demonstrable and sustainable change

Behavioural change comes before attitude change – thinking comes after the fact

In order to change attitudes, change behaviour

We need to understand who we are so we can create solutions

More rationalising than rational

Cognitive outsourcing – memory is a distributed function so only remember slivers

We are fundamentally social with a bit of independence, not vice versa

Asian culture is inherently social

Gandhi said that humans are a necessarily interconnected species

Thinking is much less important in human life than it seems

“lazy mind hypothesis”

We can think independently, we just don’t like it – like a cat to water

Behave according to other people’s actions e.g. go to busy shops

We know our own judgement is fallible so “I’ll have what she’s having” – wisdom of crowds or financial crisis

Leads to social norms

Need to design our theories and tools to accommodate social behaviour

Genesis random – Duncan watts

Is it proximity that leads us to follow a herd, or example of using rationally weighing up the pros and cons

Herds originate from somewhere – must be a leader. Are these leaders the same in each situation, or are we all capable of being shepherds

Research application – crowdsource answers. But random – no statistical certainty as only one situation

Wikis to collate group opinion?

Reblog this post [with Zemanta]

Cluetrainplus10: Thesis no.2

This is my blog post on thesis 2 of the Cluetrain Manifesto, forming part of cluetrainplus10. This is a project set up by Keith McArthur to celebrate the ten year anniversary of the manifesto’s publishing. I am one of many bloggers who has picked a thesis to cover today.

I feel like a bit of a charlatan, as I haven’t read the full book. I feel like I have, since the book gets referenced and rehashed so often but I should really go to the source at some point the get the version without embellishments and misinterpretations. I have at least read the manifesto though, and there was a thesis available that I wanted to cover so…

2. Markets consist of human beings, not demographic sectors.

Without wishing to revert to school essay-writing style, it is important to deconstruct the parts of this thesis.

Firstly, markets. Straightforward enough – an exchange of a good or service between a giver and receiver. The economy is made up of a vast number of complex and interconnected markets.

Secondly, demographic sectors. Now the tighter definition of a demographic will look at the objective population characteristics of that segment. Age, gender, ethnicity and so on.

However, loosening this could incorporate location-based, attitudinal, behavioural or lifestyle factors. Segmentation is not a science, after all.

Prisoner Patrick McGoohanThirdly, and finally, there is human beings. We have consciousness, emotions, motivations and free thought. We are not numbers, we are free men.

So, on a tight reading, the thesis could be saying that we shouldn’t be grouped into segments or demographic sectors, but treated as individuals that can fluctuate in and out of pre-defined targets as and when we please.

Technically correct, but this works better for pull-markets than push. In a pull market, the seller has ceded a degree of control. I self-select myself to customise the experience within the constraints to give myself maximum utility. The web has been a great enabler of this.

But most markets are still push markets. Unless your population is a super-select group (e.g. multi-billionaires), it is technically infeasible to treat all potential traders as individuals. That is where demographic sectors come in useful. Population characteristics are pretty outdated and completely overlook the fantastic diversity of our society. Attitudinal or behavioural demographics are much more useful (and fluid).

This reading also overlooks an important element of the thesis. As human beings we are plural. We may be individuals, but we also act in groups. Some might say that we have an inherent herd mentality.

So it is feasible to target groups by attitude, but we should treat them with more grace and humility. With humanity. Not calling them targets, for one thing.

And this works both ways. We should be human ourselves. Organisations should display this emotion, free thought and consciousness that defines us as who we are.

This gets to the heart of the thesis, in my opinion. And it is ever more relevant as the economy gets destroyed by rampant, greedy capitalism. It may not bring the short-term efficiency of a quick trade on the stock exchange or a last second snipe on ebay, but it creates meaningful and long-lasting relationships. Which ultimately benefits both sides.

We are people. We may be grouped, but we are not homogeneous. We are not faceless, we have multiple faces. Our name is legion. And we should recognise this.

We have been slowly learning to treat the customer with respect by using various platitudes.

“The consumer isn’t a moron. She is your wife”David Ogilvy

Now it is time to respect ourselves.

sk

Reblog this post [with Zemanta]
Follow

Get every new post delivered to your Inbox.

Join 35 other followers