Online surveys help brands collect consumer feedback quickly and at scale. But that feedback only has value if the responses behind it are genuine.
Guaranteeing response quality is becoming harder as survey bots become more common. These bots can complete surveys automatically and contaminate datasets before researchers even realize there’s a problem.
What makes it even worse is that they’ve become harder to detect. More advanced bots can mimic human-like patterns and even produce open-text answers that look convincing at first glance.
For businesses, the risk is significant. Bot responses can skew survey results, waste research budgets and lead teams to make decisions based on unreliable data.
To help you protect your data quality, this article explains what survey bots are, how they affect survey data, how to detect suspicious responses and what to do once those responses have been flagged.
TL;DR
In this guide, you’ll learn:
- Survey bots can submit large numbers of fake responses, distort findings, waste research budgets and undermine stakeholder confidence in research.
- No single check can reliably prove that a response came from a bot. Look for combinations of warning signs, such as rapid completion speeds, attention check failures, inconsistent answers and suspicious technical patterns.
- Open-text responses need careful review. Generic language, repeated phrasing or answers that contradict the rest of the survey may point to bot activity.
- Once a response is flagged, assess the overall weight of evidence before removing it. Borderline cases still require human review.
- Invalid responses should be removed and replaced so the final dataset remains clean.
- Attest protects every survey with automated and AI-enabled checks, human oversight and a multi-panel approach that helps identify suspicious responses while surveys are live.
What are survey bots?
The term “bot” refers to a script or program that is able to fill in the fields of a survey with fake values and then submit the survey, repeating the process many times, with the goal of receiving the promised incentives multiple times as well. Bots like this have been a problem for a long time in consumer research.
To be clear, bots aren’t an issue in every online survey. They’re more likely to affect surveys that are open to anyone with a link, especially when there is a reward attached, such as a gift card or other incentive.
How bots impact your survey data
Survey bots are especially risky because they don’t always look suspicious at first and only become obvious later when survey results start to look off. By that point, the damage may already be done and have affected analysis and, in turn, stakeholder confidence.
Here’s how bots can impact your survey data:
Distorted results lead to poor decisions
Recent research by Goodrich et al. found that fraudulent responses made up 96% of responses in one incentivized online survey and 72% in another. They also found statistically significant differences between fraudulent and valid responses, showing just how much bot contamination can distort the story your data is telling you. These figures are extreme, but they highlight the importance of using a survey provider with robust data quality checks in place.
When it comes to consumer research, bot responses may make a product concept look more appealing than it really is or add noise to close-ended question analysis. If those insights make it into a report, teams risk making decisions based on bad data.
Research costs increase
Bots can claim incentives that should have gone to genuine respondents. They also create extra work for research teams, who must spend extra time cleaning the data and rerunning part of the study.
In more serious cases, the team may need to rerun a survey, which increases research costs significantly.
Reduced stakeholder confidence in online research
Bot contamination can make stakeholders question more than one set of results. Once they see that fake responses can enter a survey, they may become more skeptical of online research overall, even when most responses are valid.
That makes survey data quality hard to treat as a background process. Research teams need to show how suspicious responses were flagged, reviewed and removed, so stakeholders can trust the findings and the process behind them.
The problem compounds quickly
Bots can submit multiple responses in a short period of time. Without quality checks during fieldwork, a small number of suspicious entries can turn into a much larger data quality problem before anyone notices.
How to detect bots in survey responses
So, how do you know if bots have made it into your survey data? There’s no single giveaway, but there are several warning signs that can help you identify responses worth reviewing before they affect your results.
Check survey completion times
One of the best ways to spot suspicious responses is to look at survey completion times. Most survey platforms record this automatically, so look out for very fast or uniform completion times.
For example, if a survey normally takes a human 10 minutes to complete but some responses come in after 30 seconds, that’s a clear red flag.
You should also keep an eye out for clusters of near-identical completion times. Real respondents move through surveys at different speeds, so their completion times should vary slightly. If hundreds of responses are completed in almost exactly the same amount of time, say five minutes, it may suggest a script is submitting responses at a set pace.
Use impossible or hidden fields
Branching logic can help you expose bot responses by creating questions that a real respondent should never be able to reach.
For example, imagine you ask a single-choice question about someone’s favorite color. A respondent can choose “red” or “yellow,” but not both. You could then create a follow-up question that only appears if someone selected both “red” and “yellow.” Since that condition can never be met by a person completing the survey normally, the follow-up question should never appear.
A bot or script may still pick up that hidden field in the survey code and submit an answer for it. So, if an impossible-to-reach question has an answer, the response should be treated as suspicious and flagged for review.
Review attention check failures
Attention checks, sometimes called trap questions, help you see whether someone is actually reading the survey. They ask respondents to choose a specific answer, such as “Please select B,” or include an obviously wrong option in a normal question, such as “soccer” in a list of ice cream flavors.
These questions can also be used to catch bots that move through surveys automatically. But they should be used carefully.
Failing one attention check may not be enough to remove a response on its own. Repeated failures, or a failed attention check alongside very fast completion times, inconsistent answers or poor open-text responses, should raise concern.
Assess open-ended response quality
Open-ended questions can help you spot suspicious responses because they ask respondents to explain something in their own words, which is harder for bots to do.
When reviewing open-text answers, look for:
- Long, generic answers: When asked “Why did you give this CSAT score?” a bot may return several polished paragraphs about the importance of customer experience without mentioning anything specific about the interaction.
- Repeated phrasing: A batch of bot responses may use the same words, structure or sentence patterns. This can suggest the answers came from a shared template rather than from individual respondents.
- Unnatural consistency: Real respondents vary in how much they write and how carefully they phrase their answers. If many responses are almost identical in length, that’s not natural. It may suggest a bot is using a fixed template or an LLM to generate answers.
- Contradictions: Keep an eye out for when an open-ended answer doesn’t match the rest of the survey response. For example, a respondent says they have never heard of a brand, but the open-text answer describes a recent purchase from that brand.
Check for any suspicious technical signals
You can also check for bot activity in a survey response’s metadata. Survey platforms and panel providers often handle these checks in the background, but it’s still useful to understand the types of signals they use. That context can help explain why a response has been flagged for review.
Common signals include:
- IP address patterns: Multiple responses from the same IP address in a short period of time can suggest automated activity or repeat participation. Some bots may also use rotating IP addresses, often through proxies or VPNs, to make submissions look like they are coming from different respondents.
- Geolocation mismatches: If a respondent qualifies for a survey in one country but their location data points somewhere else, the response may need closer review.
- Device or browser patterns: A high number of responses from the same browser setup, operating system or device type can suggest that one source is submitting multiple entries.
- Repeat participation signals: Digital fingerprinting can help identify when the same device or browser appears to be completing a survey more than once, even if the IP address changes.
Review these signals alongside response behavior and answer quality. One technical red flag may not prove bot activity, but several together should raise concern.
How to handle suspicious responses after detecting them
Finding suspicious responses is only the first step. The next step is deciding what to do with them in a way that protects data quality without removing valid responses by mistake.
As Alyssa Stringer, Director of Product at Attest, explains, “We don’t think about quality as a single pass/fail test. Every response is evaluated across multiple signals.”
That means suspicious responses should be reviewed through a consistent process before they are accepted, flagged for deeper review or removed from the dataset.
Look for clusters of quality signals
Before removing responses, define the types of issues you’ll look for. Alyssa explains that Attest evaluates suspicious responses across three broad categories:
- Impossible signals: These are things that should not be true, such as impossible demographic combinations or completion times that suggest extreme speeding.
- Improbable signals: These patterns are not impossible, but they look suspicious in context. Examples include overclaiming behaviors or excessive non-answers.
- Behavioral signals: These come from how someone moves through and responds to the survey, such as attention check performance, open-text quality, consistency and engagement.
The key is to avoid treating one issue as automatic proof of bot activity. As Alyssa puts it, “In practice, quality decisions are usually based on the overall weight of evidence rather than one unusual data point.”
“Real people get distracted, misread instructions or click the wrong option,” notes Alyssa, which is why it’s so important to not base your decisions solely on one red flag.
For example, a response completed slightly faster than average may still be valid if the open-ended answers are thoughtful and the attention checks were passed. But a response that combines extreme speeding, failed attention checks, generic open text and inconsistent answers should raise much stronger concern.
Review flagged responses manually where needed
Automated checks are essential when you’re collecting hundreds or thousands of responses, but human review still matters. This is especially true for borderline cases and open-ended answers.
Alyssa notes that “advanced AI-generated responses can pass many traditional quality checks while still being synthetic.” That’s why Attest combines automated checks with final review from its Customer Research Team.
These manual reviews can help identify responses that technically pass the rules but still feel wrong. An open-text response may answer the question but feel oddly generic or detached from the context. Another may look internally consistent but lack the natural variation you’d expect from a real respondent.
Alyssa also points to survey journeys that look “too perfect,” where every question is answered and every attention check is passed, but there’s little evidence that the respondent was genuinely engaged.
Even if the response was submitted by a real person rather than a bot, it may still indicate dishonest or low-effort participation and should be reviewed before being included in the final dataset.
Remove and replace bot responses
Once a response is judged invalid, remove it from the dataset. If that reduces your usable sample size, replace it with a valid complete so the final dataset still supports the research objective.
Document the process as you go. Keep a record of why a response was removed, which signals were considered and how replacement responses were sourced. This makes the cleanup process easier to defend if stakeholders ask how data quality was protected.
Alyssa warns that the biggest mistake teams make is treating data cleaning as a search for one “magic rule.” Instead, “The better approach is to look for clusters of quality signals.” That principle should guide removals: make the decision based on the combined evidence, not one unusual metric in isolation.
How Attest safeguards your data quality
Reviewing responses manually can work for smaller surveys. But as studies scale and bots become more sophisticated, detecting every suspicious response becomes much harder.
That’s why data quality needs to be built into the research process from the start, not treated as a cleanup task after a survey ends.
Attest is a connected consumer insights engine that helps businesses answer important consumer questions with trusted data. Every survey is protected by multiple layers of automated quality checks, AI and human oversight. These features help researchers identify and remove suspicious responses before they affect the final dataset
Here’s how Attest protects your data quality:
- Automated and AI-enabled quality checks: Every survey is screened using proprietary quality checks that look for signs of fraudulent or low-quality participation while a survey is still live. These checks include anti-bot measures, digital fingerprinting, geolocation checks, device usage analysis, attention checks and suspicious response pattern detection.
- Automatic removal and replacement: When a response fails Attest’s quality checks, it’s automatically removed and replaced with a verified respondent. This means you receive clean, high-quality data without sacrificing your target sample size or having to rerun part of the study.
- Human review and oversight: Technology is only part of the process. Attest’s Customer Operations team reviews every survey, monitors fieldwork as it runs, checks open-ended responses and carries out quality spot-checks to identify issues automated systems may miss.
- LLM-based quality checks: As survey fraud evolves, so do Attest’s quality controls. LLM-based checks analyze the entire survey response rather than individual answers in isolation. This allows Attest to identify inconsistencies and suspicious response patterns that may not be obvious from rule-based checks or human review alone.
- A multi-panel approach: Attest sources respondents from hundreds of panel providers rather than relying on a single proprietary panel. That gives researchers access to diverse audiences while allowing Attest to remove underperforming or low-quality panels without reducing respondent reach.
