Treating respondents as commodities

Treating respondents as commodities – don’t do it, kids.

Yet it happens, particularly with online surveys.

I recently had a sales call with a provider who said that their panel was no better or worse than any competitor; they sought to differentiate themselves via client management and survey aesthetics.

This experience is backed up by a pithy comment from Tom H.C. Anderson in his Linked In group (NGMR – I won’t link to it, since it is only visible to members) who said “There is really only one panel that is used by everybody. Counting panelists is like counting fish in the sea and or clouds in the sky. One day they’re being used by company X, the next by company Z & Y”

Online panels aim to be as representative as possible; thus there is little difference in their make-up and so companies compete on other grounds. Primarily, this seems to be price. This means providers are continually trying to squeeze more out of their respondents for less.

This has contributed to the commoditisation of sample (it is by no means the only reason – it is perhaps an inevitability given the need to maintain respondent anonymity and confidentiality) and the research process. The research experience is at best variable (at worst, terrible) for respondents.

Surveys are now analogous to Farmville – drones click on different parts of the screen to complete monotonous tasks for a tiny reward.

This has to change. Perhaps it will – two recent articles on Research Live have broached the topic

As a user of online panels, I know I am part of the problem. But it is the panel providers’ responsibility to protect its users. This would require coordinated action across the industry. Given that market research is regulated, this shouldn’t be an issue.

And the providers could start by treating their panel members as humans, and not commodities. Notwithstanding the inefficiencies of asking questions rather than capturing data (I’ve written about this previously in “If data is the new oil, we need a bigger drill”), some simple user experience testing could provide opportunities for easy, impactful changes.

For instance, why do surveys need to always ask demographic information? I’ve been stonewalled on this by several different companies, who say that it is “standard” (which sounds like commodity-speak) or that they need to ensure information is up-to-date. It is conceivable that a panel member may have changed their gender in the interim period between surveys, but I wouldn’t expect their ethnicity or age birthday to change. Cutting out extraneous questions can easily reduce survey length, and the burden on respondents.

This is a discussion the industry needs to have, and one I’m happy to be a part of.

sk

NB: I’m not concerned about whether they are called respondents or participants. Actions are more important than semantics.

Image credit: http://www.flickr.com/photos/baconandeggs/1490449135/

Advertisements

3 Responses

  1. I would be astonished if their age didn’t change! 😉

    (But yes, we can probably make a good guess at the rate it’s changing…)

    I think there are two other parts of this problem:

    – Do the people signing up to panels expect to be treated as humans? (i.e. have we already missed the boat in terms of recruiting non-drones)

    – Is there a way to persuade clients that there’s a benefit for them in it? I admit I don’t follow discussions on “online data quality” terribly closely but they always seem to be framed from a sampling perspective with respondent experience rarely getting a look-in.

  2. I’ve amended the text – cheers!

    Taking your first point to the extreme, we coul merge research panels with Mechanical Turk. Your second point is an interesting one. Both parties can benefit – the respondent gets a less onerous experience that they might possibly enjoy and thus take more care in, the client gets a more thoughtful response. Unfortunately, a more “thoughtful” response can’t really be proven – so what if someone takes longer writing an open-ended answer – only whether the research ultimately proved to be accurate or not

  3. Thoughtfull post — makes new methodologies which companies like Linkfluence are pioneering all the more relevant. They remove all the bias of direct feedback and carefully sample social media where people make comments based on unscripted emotion. They have just began a project with Nokia Design where I work. Worth a look…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: