How to run creative testing: best practices

Shapes lining up with holes, symbolizing the creative testing process

You’ve worked hard on your ad/website/packaging (delete as applicable). 

While good creative testing is how you guarantee its success, bad creative testing can guarantee failure. Inaccurate insights can send you in the wrong direction—and no one likes having to scrap a project and start from scratch.

We’re here to make sure your creative testing research is thorough, useful and efficient. Here’s our essential guide to running good creative testing research. 

The two methods of creative testing: sequential and monadic

There are two main methods for creative testing. 

There’s monadic testing, where you divide your respondents into the number of creatives you want to test, then ask each group to review one creative idea without them seeing any of the other ideas. This usually stops people’s answers being biased by their views on any of the other ideas they see in your survey. 

Then there’s sequential monadic testing, where you can test multiple creatives against each other, one after the other, in a single survey. You can ask the audience to compare and contrast the creatives and ask which one they prefer. 

How to run creative testing research

What types of questions should you ask in a creative testing survey?

Ideally, you should ideally ask a mixture of both quantitative and qualitative questions. But if you ask too many qualitative questions then your respondents may get tired of answering, giving you low-quality data, and there’ll be a lot of answers to read through, taking a lot longer to analyze.

Qualitative questions are generally best to use in the early, exploratory stages of creative research—getting a vibe for what type of creative you should be publishing. While quantitative questions will give you a comprehensive amount of data for finding out which creative ideas work—because the numbers are bigger, you can have a bit more confidence in their authority. 

Here are some example questions you could ask to get you started. We’ve also got more creative testing example questions to inspire your research. If you’re pressed for time, we have a creative testing survey template with all the best practices built-in.

Sequential testing survey questions

Quantitative

  • Having seen both designs, which one do you prefer?
  • Which ad makes you want to buy the product more?
  • Which ad gives you a better feeling about the brand?

Qualitative

  • Why did you choose that one? Please provide as many details as possible, we really appreciate any feedback.
  • Why don’t you like X ad?
  • How could X ad be better?

Monadic testing survey questions

Quantitative:

  • Does this ad make you want to buy the product?
  • Does this ad make you feel good about the brand?
  • What is your favorite thing about the ad?
  • In your opinion how clear was it that this advert was for [brand]?

Qualitative

  • How does the ad make you feel?
  • What don’t you like about the ad?
  • How could the ad be improved?
  • Please can you explain why you have given this answer?
  • ​​What do you think was the main message of this advert?

How to reduce bias in your creative testing: 4 best practices

You want your respondents to give you insights that are completely unbiased. Biased responses = decisions based on unreliable insights. 

To avoid bias, you need to be clear about the things you want respondents to focus on and remove other distracting elements. 

Here’s how. 

1. Remove logos and familiar branding

If you were testing different types of packaging, you could remove brand logos to make sure any preconceptions the respondent has about the brand don’t influence their responses.

It’s also worth removing any colors that might remind people of your brand. Who doesn’t recognise the Coca Cola red or McDonald’s yellow?? (You shouldn’t need to worry about this if your brand isn’t as huge as these – only think about this if your brand is synonymous with your colors.)

2. Even remove products

You might also want to remove the product name, and perhaps even the product itself, to get people’s view on the packaging alone.

Imagine you’re testing packaging for a snack or soft drink:respondents could be swayed if they knew it was their favorite (or least favorite) flavor. 

3. Test creative you know doesn’t work

This might sound counterintuitive, but you’ll thank us later!

By testing old or ineffective assets, you give yourself a benchmark to compare your new assets against. So, if you get similar results as the old asset, it’s not something you can justifiably launch. But if your new research shows improvements in key areas, that’s a green light!

Pro tip 💡

“Don’t tell anyone, but it’s totally fine to test your competitors’ assets. In fact, we advise brands to make it standard practice.

By doing this, you have another benchmark to compare your assets with. But you also get an understanding of your competitors’ creative successes or failures.” 
Sam Killip
VP Customer Success

4. Test an exaggerated version of your creative

If there are certain elements of your assets you want opinions on, exaggerate or highlight them.

For example, if you’re testing packaging and want to know what people think about your imagery choices, you could make the imagery more prominent on the version you test. Or you could remove other distracting elements so that people focus more on the elements you want them to see. 

Best practices for setting your creative testing audience

Here are some pointers on how to reach the right people with your creative testing.

Work out who’s in your target audience

You should make sure your creative testing audience represents your target audience. 

Pro tip 💡

“Do you have doubts about who you should be running creative testing to? You’ll need to run some consumer profiling to get a proper sense of who’s in your target audience.” 
Elliot Barnard
Customer Research Principal

Widen your audience, and narrow it down later

You should slightly expand the parameters of your target audience for creative testing. This will allow you to compare the reaction of your non-target audience with that of your target audience. If the reactions are broadly similar, then you might find it worthwhile to expand your main target audience in the future.

For example, if your target audience is 30- to 40-year-olds, then conduct your test on 25- to 45-year-olds. 

Set audience quotas

Put audience quotas in place, especially if you are conducting monadic testing. Audience quotas are where you pre-set certain amounts of respondents from specific groups to make sure you have a good spread of responses. 

If you don’t set audience quotas and only use nationally representative samples, you could end up with different audiences reviewing different creatives, and this could mean your results are inaccurate. Putting quotas in place means you can be sure that similar, target customers audiences are reviewing your creatives, rather than a bunch of irrelevant people.

Randomize!

Make sure you randomize your creative assets (if you’re testing more than one at a time). If you don’t, then order bias might mean you end up with skewed results. 

If you randomize, then you’ll get reliable insight into how effective each of your assets is.

Test with your own audience too

While you would typically carry out creative testing on people who fit your target audience—AKA prospective customers—it can also be a good idea to carry out creative testing on your loyal customers as well. 

You can safely assume that your existing customers know your brand better than most people. That familiarity can mean that existing customers give you additional insights you hadn’t thought of before. For example they could tell you whether the creative you show them matches the brand purpose and image that they already have an understanding of.

Know when to move on

Although it’s useful to keep testing and iterating your creative ideas throughout the process of their creation (and even after they’ve been published, to a certain extent), you need to know when to stop! 

If your new creative ideas are out-performing previous creatives that you tested in the past, then there’s no need to keep tweaking and testing. If the reactions are positive overall, then you’re probably ready to launch your creative ideas to the world for real.

And if you focus too much on tweaking existing ideas, that might not give you the space to test the game-changing, off-the-wall ideas that might take your brand to the next level.

Before you launch your creative testing project, make sure you know the key mistakes you should avoid. 

The Consumer Research Academy is brought to you by the Customer Research Team—our in-house research experts. Any research questions? Email or chat with the team.