by Scott Keeter, Michael Dimock and Leah Christian, Pew Research Center for the People & the Press
Current polling in the 2008 presidential election shows a very tight race between Democrat Barack Obama and Republican John McCain. In part because of the strong support Obama is attracting among younger voters, and as the number of Americans who are reachable only by cell phones rises, interest continues to grow in the question of whether public opinion polls that do not include cell phones are accurately measuring the relative levels of support for the two candidates.
The Pew Research Center for the People & the Press has conducted three major election surveys with both cell phone and landline samples since the conclusion of the primaries. In each of the surveys, there were only small, and not statistically significant, differences between presidential horserace estimates based on the combined interviews and estimates based on the landline surveys only. Yet a virtually identical pattern is seen across all three surveys: In each case, including cell phone interviews resulted in slightly more support for Obama and slightly less for McCain, a consistent difference of two-to-three points in the margin.
For example, in Pew’s latest poll, conducted Sept. 9-14 with 2,509 registered voters, including 549 reached by cell phone, 46% backed Obama and 44% backed McCain. Among the landline respondents, the candidates were tied at 45% each. The same 2-point differential is seen if the analysis is restricted to likely voters – in this case, the candidates are tied in the combined sample, while McCain has a two-point lead among landline respondents.
Similar differences were seen in August and June. In August, Obama led by 3 points (46%-43%) in the combined sample of registered voters, while the landline sample showed the race tied at 45% each. In June, Obama led by eight points (48% to 40%) in the combined sample, and by five points (46% to 41%) in the landline sample alone. In all three cases, the overall horserace differences between the landline and combined samples are not statistically significant, but adhere to the same pattern.1
As implied by these results, in each of the three polls, the cell-only respondents were significantly more supportive of Obama (by 10-to-15 percentage points) than respondents in the landline sample. For example, in the September survey Obama led McCain by a 55%-to-36% margin among cell only voters, but the candidates were tied at 45% in the landline sample.
In large part, this reflects the fact that a substantial minority of the cell-only sample is younger than 30 – a demographic group that has consistently backed Obama this year. Traditional landline surveys are typically weighted to compensate for age and other demographic differences, but the process depends on the assumption that the people reached over landlines are similar politically to their cell-only counterparts. These surveys suggest that this assumption is increasingly questionable, particularly among younger people.
A Closer Look at Young People
To analyze differences in young people reached in a landline survey with those who are cell-only, the samples for the August and September surveys were pooled to increase the number of young people available for analysis. In the pooled data, cell-only young people are considerably less likely than young people reached by landline to identify with or lean to the Republican Party, and even less likely to say they support John McCain. Among landline respondents under age 30, there is an 18-point gap in party identification – 54% identify or lean Democratic while 36% are Republican. Among the cell-only respondents under age 30, there is a 34-point gap – 62% are Democrats, 28% Republican. The difference among registered voters on the horserace is similar: 39% of registered voters under 30 reached by landline favor McCain, compared with just 27% of cell-only respondents. Obama is backed by 52% of landline respondents under 30, compared with 62% of the cell-only.
These kinds of differences can lead to substantially different estimates of the youth vote. Weighted data from the August and September combined surveys show Obama with a 27-point lead among voters ages 18-29. A weighted estimate derived solely from young people reached by landline would show an Obama lead of 15 points. There are no differences between the combined and landline sample estimates for any of the older age groups.
Young voters may play a critical role on Election Day. But will cell-only young people turn out at the same rate as those with a landline phone? While 18-29-year-olds reached by cell phone tend to have less experience voting than their landline counterparts, they are just as interested in the 2008 campaign, and express just as much intention to vote this year. The clearest difference is on past voting behavior. Just 23% of cell-only young respondents say they “always vote,” compared with 41% among the landline respondents. There are small – and statistically non-significant – differences in the share who voted in the 2004 election and who have previously voted in their precinct. Yet at the same time, most cell-only young people are registered to vote, have given a lot of thought to the election, and say they definitely will vote – factors that are also closely associated with turnout.
Conclusion
Even though the omission of cell phones from election polls does not currently make a large difference in the substantive results, Pew’s surveys this year suggest at least the possibility of a small bias in landline surveys. Such a bias could be consequential in an election that appears to be very competitive right now, especially if significant numbers of young people turn out to vote. In particular, the research suggests that estimates of the candidate preferences of young voters may be biased if cell phone interviewing is not included in the survey.
These problems are all the more pressing as the number of Americans who are reachable only by cell phones increases. U.S. government surveys estimated that about 15% of adults were “cell only” in the fall of 2007 and the rate of increase since 2004 has been at least 2% a year, meaning that the number may be as high as 17% in this election cycle.
Many polling organizations are including cell phones in their samples this fall, and the Pew Research Center for the People & the Press will include cell phone samples in all of its remaining election polls.
For a broader discussion of the cell phone issue see these earlier Pew Research Center studies:
The Impact of “Cell-Onlys” on Public Opinion Polling
Research Roundup: Latest Findings on Cell Phones and Polling
Notes
1 In all cases, comparisons are between the weighted combined survey results and the results of the landline interviews; each sample is weighted independently to the same demographic targets. The significance test accounts for the overlapping cases in these samples.
