February 16, 2017

A basic question when reading a poll: Does it include or exclude nonvoters?

The early days of a new presidential administration produce not just a blizzard of news but a blizzard of numbers. Pollsters of all stripes race to get and report Americans’ first impressions of their new president. But, frustratingly, those reports don’t always match up as precisely as the Type A among us might wish.

Take the past three weeks of polling on President Donald Trump. Depending on the poll, Trump’s approval rating between Feb. 5 and 13 could have been as high as 53% or as low as 39%. So which was it?

There are a number of possible reasons for polls arriving at different estimates – from the mode used to collect data to how people are selected for a survey – but here we’ll tackle one of the most basic: Did the poll include or exclude the 45% of adult Americans who didn’t cast a vote last November?

Typically, polls in the U.S. are designed to represent one of three populations. The broadest is the general population of all adults (GP). Surveys based only on adults who are registered to vote (RV) apply a narrower lens on the public. Narrower still is the filter applied with surveys that interview only registered voters who are deemed likely to vote (LV). Many pollsters might conduct surveys of all three, depending on where they find themselves in the election cycle.

In non-election years like this one, most pollsters survey all adults, but not all follow this convention. A number of pollsters continue to do surveys of registered or even likely voters. Why does this matter for Trump’s approval ratings? It’s about demographics. Voters as a group skew older and whiter than the general public. And older Americans, as well as white Americans, tilt more Republican than other groups. So, voter-only polls tend to get somewhat more favorable views of a Republican president or candidate and less favorable views of Democrats. This pattern was evident during Barack Obama’s presidency, with his overall ratings tending to be somewhat higher among the general public than among registered or likely voters. 

A look at some of the presidential approval numbers released this month shows a pattern consistent with these demographic differences. LV polls – those surveys based only on the views of “likely voters” – are generally reporting higher levels of support for Trump than general population polls. There is a more muted but still significant difference in an RV-GP comparison: Pew Research Center’s general population poll conducted Feb. 7-12 recorded Trump’s presidential approval rating at 39%. Among registered voters in that survey, his rating was 42%.

Registered voter polls exclude nearly 40% of the population

So how many people are we including, or excluding, when we decide to look at the views of only registered or likely voters? In the U.S., roughly six-in-ten adults are registered to vote. The registration rate tends to tick up a few points in presidential election years (to about 65%) and back down a few points in non-election years. This means that RV polls, by design, exclude nearly 40% of adults living in the U.S.

LV polls, of course, exclude even more, as they aim to include only those registered voters who actually cast, or will cast, a ballot. The voting rate among adults ages 18 and older was 55% in the 2016 presidential election and 33% in the most recent midterm election. If a pollster is currently conducting an LV poll with 2018 (a midterm election year) in mind, then recent midterm voting behavior suggests that their results might be excluding the views of a majority of adults in the U.S.

So what is the appropriate population to represent?

Which population is appropriate – all adults, registered voters or likely voters – depends on what information the poll is intended to gather. For political operatives whose jobs (winning elections) require them to be in perpetual campaign mode, polls of registered or likely voters arguably provide the most accurate information about the mood and opinions of that subset of Americans who will actually cast a ballot in the next election. The fact that the poll excludes a large share of the population is a desired feature, not a bug.

For those interested in the opinions and experiences of the entire country, as is the case here at the Center, general population polls are a more informative and useful tool. Nonvoters, like any other group of Americans, include people who pay taxes, want health insurance, run small businesses, use the public education or court systems and have a stake in the nation’s security – even though they do not play a role in selecting the officials who craft the public policies that affect them.

A last thought on nonvoters

While nonvoters do lean more Democratic than Republican, it would be a mistake to cast them as a partisan lot. Overall, they are much less likely than registered adults to identify with either major party (24% don’t lean toward either party, compared with 6% among voters). So polls that exclude nonvoters don’t just change the shade of the country’s partisan balance, they paint a portrait of a public that is distinctly more politically engaged and ideological than those that survey all adults.

Correction: A previous version of this post and accompanying chart gave an incorrect figure for the Feb. 7 Rasmussen poll.

Topics: Donald Trump, Political Attitudes and Values, Polling, Research Methods, Telephone Survey Methods, Web Survey Methods

  1. is director of survey research at Pew Research Center.

  2. Photo of Claudia Deane

    is vice president of research at Pew Research Center.

6 Comments

  1. Chris Pederson1 month ago

    hey it’s that guy who was wildly inaccurate about the 2016 election, I remember you

  2. Anonymous1 month ago

    Actually many of Trump supporters never voted before, and were missed in the polling before the election so how do you account for that?

  3. Anonymous1 month ago

    Thank you very much for this article. I think it’s incredibly important to take into account the non voting population (the general population) when commenting on policies and actions concerning this administration. In my younger years, I admit to being among the non-voting class, principally because I believed that one party was likely to win vs the other and/or whichever party won was likely to maintain the status quo with little disruption to my day-to-day life. I think many of the potential voters in this last election perceived a likely win by Clinton, and therefore did not vote as they did not perceive their status would change appreciably. I also believe, judging by the amount of resistance, that these individuals are now ‘woke’ and are actively participating in government in the form of anti-administration activities. It might be interesting to poll the subset of the population that did not vote to test this hypothesis.

  4. Anonymous1 month ago

    It would have been interesting to also compare the same metrics for other recent presidents.

    John Stoesser
    Barrington, IL
    jstoesser@gmail.com

  5. Willilam Magill1 month ago

    So what is the appropriate population to represent?

    The correct answer is — all 3. Reporting only one of the 3 groups biases the results; and the interpretation of those results- and clearly in today’s environment that is an important issue.

    You have stated that “non-voters lean more Democrat than Republican.” then try to say “24% don’t lean toward either party, compared with 6% among voters” — which is it?

    Equally important is the geographic distribution of your 1000 respondents.
    No poll of 1000 or even 5000 individuals can hope to ACCURATELY convey the attitudes of 3 million! Randomizing 1000 folks across all 50 states (easier to divide) implies that there should be 20 from each state — or it might mean that there are 50 from California and none from Pennsylvnia.

    As those of us involved in computer gaming know — Randomizing is the easiest way to alienate your customer base. It doesn’t matter how mathematically equitable it is it is always seen as favoring the person who “got theirs” (when I didn’t get mine).

  6. Anonymous1 month ago

    Excellent piece. I’m of the belief that those who don’t vote have opinions that don’t count. They shouldn’t be included in any polls on approval ratings.

    At the end of the day, less than half of the 55% of 18+ adults in America who voted selected Hillary and even fewer selected Trump. Given it was such a polarizing election, we knew the majority of Americans would be unhappy with the result either way.

    So then, it’s no wonder approval ratings are what they are. Let’s give it some time and see what direction it goes.