Q. I’m hoping you can answer a question I have about the accuracy of current polling practices. I believe a high percentage of people now have phone systems that let them know who is calling before they answer the call. If others are like me and my friends, we simply don’t answer calls from strangers, including pollsters. This is a fairly recent development. Have you found an increased incidence of “non-response” and if so, how do you balance it in your samples?
You are certainly not alone in ducking our calls (but please consider picking up sometime – you might enjoy the interview). Survey non-response has been increasing steadily over the past two decades. Voice mail and caller ID have made it easier for people to avoid calls from unknown sources, but we also find that more people are refusing to be interviewed even when they do answer. This is true not only for telephone surveys but for personal interviews where the request is made face-to-face.
This trend has certainly added to the difficulty and cost of conducting high-quality survey research, but it is not the case that non-response necessarily causes polls to be less accurate. One yardstick for judging this is how polling does in forecasting the outcome of elections. If the kinds of people who refuse to take part in surveys are different from those who do participate, our polls will be biased. Yet, in the last several election cycles, most national telephone polls (including ours) have been very accurate. The National Council on Public Polls compiles the election forecasts of the major national polls, and in both 2004 and 2008, these estimates were very good predictors of the final vote.
One reason that polling can overcome the problem of low response rates is that most pollsters — including Pew Research — weight their data to ensure that the demographic composition of the samples conforms to the known national parameters for such characteristics as sex, age, race, education and region. Here is more detail on how we weight our data (and on our methods more generally).
Scott Keeter, Director of Survey Research, Pew Research Center
