There are good arguments against weighting surveys to match how people voted in past elections or an independent estimate of political party affiliation. We know because we’ve made many of those arguments over the years! However, the steep decline in survey response rates and the rise in differential partisan nonresponse have made those arguments less accurate or relevant today.
This report summarizes the evidence that weighting on past vote or party affiliation in the current era tends to improve accuracy over just weighting on core demographics like age, gender and education. It is worth highlighting the limits of this practice, as it is not a panacea.
(This is part of a larger Pew Research Center report about political weighting – adjusting a survey to reflect the political composition of the United States as accurately as possible. For an overview and the report’s key takeaways, read “Weighting Polls on Party Affiliation and Past Vote.”)
Does weighting on past vote or party affiliation improve all estimates in a poll?
No. Pew Research Center studies finding support for this practice focus on the average accuracy level across a range of estimates. We find that, on average, weighting on party affiliation improves accuracy. But there are many questions where it makes no meaningful difference, and a small number of questions (such as personal finances) where it can lead to somewhat lower accuracy than just weighting on core demographics.
Does weighting on past vote or party affiliation make estimates perfectly accurate?
No. Weighting on party affiliation tends to make political poll estimates more accurate, but it does not make them perfect.
One sobering finding was that having an accurate share of Republicans was not sufficient for estimating Donald Trump’s 2020 vote share accurately. The American Association for Public Opinion Research’s 2020 election postmortem found that “Polls weighted by partisanship and 2016 past vote were not more accurate, on average, than those weighted using only standard demographics. Weighting on partisanship and past vote may have reduced polling error, but it did not solve the issue.” The implication was that weighting up Republicans to be represented proportionally does not work if the Republicans in the poll are not representative of Republicans in the population.
Should all surveys weight on political balance?
Probably not. This practice makes sense for low-response-rate surveys probing political attitudes and behaviors. It may not be helpful for surveys probing topics that have nothing to do with politics. There is a statistical reason for this. A weighting adjustment improves an estimate if the adjustment measure (e.g., party affiliation) is related both to who participates and to the key concept (e.g., vote choice) being measured in the survey. If these things are not related, the weighting adjustment will not do any good.
At Pew Research Center, we find partisanship is associated with many topics we study. There are partisan differences in everything from religion and news consumption to attitudes about science and technology. It makes sense, then, for us to use a weighting adjustment to ensure we have the right shares of Republicans and Democrats in our polls. For researchers studying topics wholly disconnected from politics, it may be unnecessary to weight on party affiliation or past vote.
On a cautionary note, some surveys may appear nonpolitical but still contain outcomes related to politics. For example, some health-related polls significantly overestimated COVID-19 vaccination rates in 2021. Those surveys did not weight on party affiliation. Considering that Democrats were significantly more likely than Republicans to be vaccinated for COVID-19, the absence of that weighting adjustment may very well have contributed to the overestimation error.
One other potential benefit of weighting on past vote is that this weight typically ensures that the relative share of voters and nonvoters in the sample reflects the population. Surveys typically overrepresent voters because they are more interested in politics and more willing to take a survey about the topic.
Should pollsters weight on party affiliation forever?
Probably not. Weighting adjustments reflect shifting patterns in how the public relates to surveys. Currently, there are both political reasons and structural factors for why polls tend to have a harder time reaching Republicans than Democrats. But, just as this has not always been the case, those dynamics could change in the future. It’s possible that several years from now Republicans and Democrats will be about equally likely to respond to a poll. If that happens, weighting on party affiliation may be unnecessary.
Does this mean pollsters should give up trying to get representative samples?
No. The goal of public opinion polling is to accurately represent the voices and views of the public – not just the people who are easiest to reach or most likely to respond. That means obtaining samples that reflect the full diversity of the population remains a core principle of high-quality polling.
Weighting on a political outcome can improve accuracy, but it does not fully remove the error in polls. That’s why rigorous polling still requires pollsters to follow established best practices, including recruiting representative samples and making participation easy and attractive for people from all walks of life.
Does it matter how the survey was conducted?
Yes. Weighting on party affiliation makes the most sense for typical polls. Most political polls tend to have very low response rates, and there is evidence that those who refuse to respond differ politically from those who participate.
High-response-rate surveys, especially those offering offline response options, may not need to weight on party affiliation. To take an extreme example, we are not aware of any evidence that the Census Bureau’s American Community Survey – a $250 million survey with compulsory participation and a roughly 83% response rate – should be weighting on party affiliation.
Does weighting on a political outcome always work?
No. Any weighting approach, like a partisanship adjustment, may not improve accuracy in all situations. Indeed, if a pollster forces a poll’s current political makeup to match an outdated benchmark, errors could increase.
Shouldn’t polls themselves be able to produce an accurate estimate of party?
According to this argument, a pollster’s current poll is the best available measure of the shares of Republicans and Democrats in the country. Aligning the poll with an external benchmark of party affiliation is unnecessary. An important piece of this argument is that “yes, polls need to be weighted – but only on core demographics like age, race, education, geography and gender.”
That view dominated the industry for decades, and it appeared justified by polling’s generally good record in forecasting elections. Even as response rates declined in the 1990s and 2000s, people’s political views did not seem to affect their willingness to participate in polls.
Fast forward to today, and there is mounting evidence that is no longer true. One study found that Democrats were significantly more likely than Republicans to participate in 2020 preelection surveys – even after controlling for factors such as age, gender, race and urban vs. rural. Other studies, including one from Pew Research Center, found that weighting just on core demographics yielded less accurate estimates than when an adjustment for party affiliation was included.
This raises the question of why adjusting on core demographics no longer suffices. The answer is that the traditional demographic characteristics no longer reliably identify who is refusing to take surveys these days. For example, if the White men ages 50 and older who participate in a poll are more Democratic than those who refuse, then the traditional approach of just weighting on things like race, gender and age falls flat. The poll will have a Democratic bias even after demographic weighting.
Party affiliation is an attitude. Doesn’t that make it a poor weighting variable?
It’s true that weighting on party affiliation is more challenging because it is fundamentally an opinion – and like any opinion, it can change. This prospect of change makes timing more critical for party weighting than weighting on more stable outcomes or ones that change slowly and predictably, such as the population’s age distribution. Pollsters weighting on party affiliation need to time their survey measure as closely as possible to the benchmark. As explained in an earlier section of this report, this timing can be much more practical for surveys fielded on a panel than for one-time (or “cross-sectional”) surveys.
While the attitudinal nature of party affiliation makes weighting more challenging, it is not impossible. Most importantly, there is compelling evidence (presented earlier in this report) that political polls tend to be more accurate with such an adjustment than without one, despite the challenges.
