How it works
Data quality
Our audience
Expert support
Features
By Use Case
Brand tracking
Consumer profiling
Market analysis
New product development
Multi-market research
Creative testing
Concept testing
Campaign tracking
Competitor analysis
Quant & qual insights
Seasonal research
By Role
Marketers
Insights professionals
2024 UK media consumption report
2024 US media consumption report
Consumer connection playbook
AI in the shopping experience
Consumer Research Academy
Survey templates
Help center
Blog
Careers
You want respondents to tell you the truth.
Hearing the truth might hurt sometimes. It might take your marketing, product or sales in a completely new direction. But you’ll be thankful later when you have a product or service that is EXACTLY what consumers want.
It’s therefore super important that you remove any chance of biasing your respondents’ answers.
Here are a few simple and effective ways to avoid bias in your surveys.
And here’s the TL;DR list to kick you off:
Try to avoid questions with yes/no answers. This is because respondents have a tendency to want to say yes—it’s the positive, aspirational answer. This is called acquiescence bias.
For example, instead of “do you find beautiful packaging essential when purchasing dark chocolate?” Ask, “which of the following do you find essential when purchasing dark chocolate?”
Look out for leading questions, which push respondents to respond in the way you want them to. This sometimes results in confirmation bias. You can do this by avoiding unnecessary adjectives.
For example, don’t say “What do you think of [brand’s] delicious dark chocolate”. Instead, say “[brand’s] dark chocolate.”
And don’t forget that scale question answers can also be leading. Make sure your answers for scale questions include positive and negative options, not just all positive options.
The way you order your questions and answers has an effect on how people respond. This is called order bias.
Order bias can happen when a question you’ve asked (or a media asset you’ve presented) has an impact on how your respondents think about the rest of the survey.
For example, if you’re presenting a series of adverts, all subsequent adverts will likely be compared to the earlier ones in the respondent’s mind, even subconsciously.
Here’s what you can do to avoid order bias in questions:
The answers people give can also be influenced by the order in which you present them. People are more likely to select the answer(s) towards the top of the list. If this happens, you won’t have a true representation of your audience’s views.
Any good survey platform will allow you to randomize answer options. It makes sense to do this for all answers to all questions—except when you’re using scale answers (e.g. very likely, likely, neither likely or unlikely…).
Don’t give away who you are! If you do, this can cause sponsor bias—when respondents feel they need to give positive answers about the brand asking the questions.
And it also stops people from letting their perceptions of your brand cloud their answers. Yes, you need to know what consumers think about your brand—that’s why we go on about brand tracking!—but there are times when you want to know what people think regardless of your brand.
For example, if you’re running new product development research, you want to know what people think about that exciting new product or service you’re considering. You don’t want their opinions about your brand—positive or negative—to influence the product insights.
Surveys are often designed to flow from the most general topics to the most specific. This is to make sure certain ideas or feelings aren’t brought to the front of mind because they were mentioned in previous questions.
If possible, try to avoid emotionally-loaded language (words and phrases with positive or negative descriptors beyond their literal meaning) entirely. Be as neutral as possible.
Avoid confusing respondents, or leaving room for interpretation in answers by objectifying your language.
For example, instead of using ‘recently’, ‘often’ or ‘infrequently’, specify the time frame as ‘daily’, ‘weekly’, or ‘once per year’.
And don’t assume that respondents understand your business jargon, acronyms or slang. Make sure you know how your ideal customer speaks—this should help you use the right language.
Avoid making an assumption that forces respondents to answer untruthfully. You can correct this easily if you consider whether or not you should include the options ‘None’, ‘N/A’ and ‘Other’ in your list of answers.
For example, asking ‘Which type of laptop do you own?’ assumes that the respondent owns a laptop, so you’ll want to make sure they can answer truthfully by adding options for ‘None’ or ‘N/A’.
And look out for places where you’ve assumed a positive or negative response, by wording your questions to allow for either option.
Instead of ‘What did you enjoy about your last holiday?’, reword this as two distinct questions: ‘What did you think about your last holiday?’ ‘Very enjoyable’, ‘Quite enjoyable’ etc. And remember to add an option for the response ‘I haven’t taken an international holiday [in the last year].’
Then follow this up with a free text question like ‘Why do you feel this way?’
The Consumer Research Academy is brought to you by the Customer Research Team—our in-house research experts. Any research questions? Email or chat with the team.