June 17, 2013

Political views can drive wrong poll answers, unless there’s money involved

Do one’s political leanings blind people to the facts? Judging from decades of polling, the answer is sometimes yes—partisanship appears to lead some people to claim as facts things that aren’t particularly factual.

For example, national surveys have found that Democrats are more likely to say that inflation rose under Ronald Reagan while Republicans say that the deficit rose during the Clinton administration. Some 81% of Democrats but 33% of Republicans in an April 2011 New York Times/CBS News poll said Obama was born in the U.S.

What explains these partisan gaps?  Political scientists from Yale University and the University of California-San Diego think they may know. Their working paper released last month by the National Bureau of Economic Research found significant numbers of Democrats and Republicans actually know the correct answer to these questions—or know that they don’t know—but choose to give pollsters the incorrect response in order to cast their party in a favorable light or stick it to the opposition.

In fact, the researchers found that they could cut the partisan gap by more than half if they offered survey respondents a chance at a monetary reward for answering questions correctly.  And when they gave respondents incentives to acknowledge that they don’t know the right answer, the partisan knowledge gap nearly disappeared.

In a 2008 national internet survey conducted by the YouGov and Polimetrix research firms, investigators randomly assigned 419 self-described Republicans and Democrats to two groups.  Those in the control group were asked 12 knowledge questions, including whether the unemployment and inflation rates had gone up or down under President George W. Bush and factual questions about the wars in Iraq and Afghanistan.  As expected, large partisan gaps on nine of the questions were found among respondents in the control group.

The second group was asked the same questions but was told that for every correct answer, their name would be entered into a drawing for a $200 Amazon.com gift certificate.  Respondents also were told that the more right answers they got, the better their chances of winning.

Oh, how the prospect of a payoff can sharpen the mind:  Among those offered the monetary incentive, the partisan gaps fell on average “by about 55% and 60% across all of the questions for which partisan gaps appear,” the study concluded.

In a second experiment conducted in 2012 with larger samples of Republicans and Democrats, respondents were offered incentives if they gave the right answer and also if they said they did not know.  This time the gap was reduced by 80%.

“The experiments show that small payments for correct and ‘don’t know’ responses sharply diminish the gap between Democrats and Republicans in responses to ‘partisan’ factual questions,” the paper concluded. “The results suggest that the apparent differences in factual beliefs between members of different parties may be more illusory than real.”

Political Typology Quiz: How Red or Blue are You? 
Choose from a series of paired statements to determine where you fit on the political spectrum. Take the quiz.

Category: Social Studies

Topics: Political Attitudes and Values, Political Party Affiliation

  1. Photo of Rich Morin

    is a senior editor focusing on social and demographic trends at Pew Research Center.


  1. Ben Hyde4 years ago

    I wonder a bit if this just reveals that people are aware of the “consensus elite opinion” and in exchange for cash will cough up that version of reality. I.e. that what the researchs were asking was not “the truth” but “your estimate of what the research time thinks the truth,” or “pretend you are in my tribe and tell me what we think.”

  2. Jack4 years ago

    Just wanted to point out that the factcheck link showcasing the CBO budget surplus numbers is patently wrong. The was no surplus as evidenced by the underlying debt holdings. Read the discussion: youargue.com/index.php/50pw13st26

  3. Zack4 years ago

    A potential confound is the fact that people who believe in conspiracy theories recognize that they are holding minority views. Just like one would disingenuously flatter a dictator to gain favor or avoid punishment, a person trying to get the “correct” answer for a reward will play to the perceived bias of those who set up the questionnaire. In other words, we cannot tell whether the monetary rewards were resulting in dishonesty or preventing it.