October 17, 2008

Poll Power

by Scott Keeter, Director of Survey Research, Pew Research Center

As the votes were counted on the night of this past January’s New Hampshire Democratic presidential primary, pollsters and other professionals in the political game began to grapple with an uncomfortable fact: Virtually all of them had been dead wrong. Despite unanimous poll results predicting a Barack Obama victory (by an average of eight percentage points) on the heels of Sen. Obama’s surprising triumph in the Iowa caucuses, Hillary Clinton was going to emerge the winner.

The New Hampshire debacle was not the most significant failure in the history of public-opinion polling, but it joined a list of major embarrassments that includes the Florida exit polling in the 2000 presidential election, which prompted several networks to project an Al Gore victory, and the national polls in the 1948 race, which led to perhaps the most famous headline in U.S. political history: “Dewey Defeats Truman.” After intense criticism for previous failures and equally intense efforts by pollsters to improve their techniques, this was not supposed to happen.

New Hampshire gave new life to many nagging doubts about polling and criticisms of its role in American politics. Are polls really accurate? Can surveys of small groups of people give a true reading of what a much larger group thinks? What about bias? Don’t pollsters stack the deck?

At a deeper level, the unease about polling grows out of fears about its impact on democracy. There is a suspicion that polls (and journalists) induce political passivity by telling Americans what they think. At the same time, some worry that polls put too much power in the hands of an uninformed public, and that they reduce political leaders to slavish followers of public opinion.

But while there may be reason to worry about the public’s political competence, a far more serious threat to democracy arises from the large disparities in income, education, and other resources needed to participate effectively in politics. Compared with most other Western democracies, the United States has a more pronounced class skew in voter turnout and other forms of political participation, with the affluent much more politically active than those who are less well off. This uneven distribution of political engagement is what makes public-opinion polls especially valuable. Far from undermining democracy, they enhance it: They make it more democratic.

Whatever their pitfalls, election polls face the ultimate measure of accountability: reality. By that standard, their track record is very good. In 2004, nearly every national pollster correctly forecast that Bush would win in a close election, and the average of the polls predicted a Bush total within a few tenths of a percent of what he achieved. Among statewide polls in races for governor and U.S. Senate, 90% correctly forecast the winner, and many that did not were still within the margin of sampling error. The record in 2000 was similar, though that was an even closer election.

The eminent political scientist V. O. Key once defined public opinion as “those opinions held by private persons which governments find it prudent to heed.” Though by no means a perfect instrument, polls make it possible for more opinions, held by a broader and more representative range of citizens, to be known to the government and thus, potentially, heeded.

Read the full analysis of strengths and weaknesses in modern day polling and what they may imply about survey-based predictions related to this November’s election.

This excerpt is reprinted, with permission, from the Autumn 2008 Wilson Quarterly