Note: For Pew Research Center’s 2023 findings on online polling accuracy, read Comparing Two Types of Online Survey Samples.
Q. How statistically accurate is an online poll in which participants sign on to contribute their opinions? Would it be possible to get statistically accurate opinions about what the average American wants their taxes to go to through an online poll that allows the participant to indicate what percentage of his/her taxes would go for each of a number of purposes?
The accuracy of a poll depends on how it was conducted. Most of Pew Research’s polling is done by telephone. By contrast, most online polls that use participants who volunteer to take part do not have a proven record of accuracy. There are at least two reasons for this. One is that not everyone in the U.S. uses the internet, and those who do not are demographically different from the rest of the public. Another reason is that people who volunteer for polls may be different from other people in ways that could make the poll unrepresentative. At worst, online polls can be seriously biased if people who hold a particular point of view are more motivated to participate than those with a different point of view. A good example of this was seen in 1998 when AOL posted an online poll asking if President Clinton should resign because of his relationship with a White House intern. The online poll found that 52% of the more than 100,000 respondents said he should. Telephone polls conducted at the same time with much smaller but representative samples of the public found far fewer saying the president should resign (21% in a CBS poll, 23% in a Gallup poll, and 36% in an ABC poll). The president’s critics were highly motivated to register their disapproval of his behavior, and this resulted in a biased measurement of public opinion in the AOL online poll.
The American Association for Public Opinion Research recently released a detailed report on the strengths and weaknesses of online polling. The full text of the report can be found here. Keep in mind that there is nothing inherently wrong with conducting surveys online; we do many of them with special populations such as foreign policy experts and scientists. And some online surveys are done with special panels of respondents who have been recruited randomly and then provided with internet access if they do not have it.
You also ask whether online polling could determine how Americans would like to see tax revenue spent. Theoretically it could. Because of the complexity of the federal budget, any such poll would have inherent limitations, of course. And an online poll that people volunteered for would have the same potential flaws as other online polling.
Scott Keeter, Director of Survey Research, Pew Research Center
