Survey researchers have long been concerned about respondents who answer questions inattentively. For traditional probability-based surveys, such respondents typically make up only a small fraction of those sampled. The process of selecting respondents at random from a list containing the entire population of interest means there is no way for potential bad actors to self-select into probability-based samples.
For online opt-in samples, individuals can sign up to take as many surveys as they want, collecting a small reward for each one. This creates an incentive for people or organizations to take as many surveys as possible in order to maximize their reward. As a result, online opt-in samples have been flooded with “bogus respondents” who make no effort to answer questions truthfully and instead try to complete surveys as quickly as possible without being disqualified.
The presence of bogus respondents in opt-in samples can lead to large survey errors, particularly on questions about rare attitudes and behaviors. Bogus respondents also disproportionately claim to be either young or Hispanic, resulting in dramatic survey errors for these groups that were, on average, nearly twice as large as the error for the general population in a 2023 Pew Research benchmarking study.
This could be because young and Hispanic respondents are highly sought-after by researchers, and by claiming to be members of these groups, bogus respondents increase their chances of qualifying for surveys. It could also be due to question format. For example, Hispanic ethnicity is frequently measured using a yes/no question, and bogus respondents have been found to answer such questions in the affirmative – regardless of their topic.
