by Scott Keeter, Leah Christian and Michael Dimock, Pew Research Center
The number of Americans who rely solely or mostly on a cell phone has been growing for several years, posing an increasing likelihood that public opinion polls conducted only by landline telephone will be biased. A new analysis of Pew Research Center pre-election surveys conducted this year finds that support for Republican candidates was significantly higher in samples based only on landlines than in dual frame samples that combined landline and cell phone interviews.
The difference in the margin among likely voters this year is about twice as large as in 2008.
Across three Pew Research polls conducted in fall 2010 — conducted among 5,216 likely voters, including 1,712 interviewed on cell phones — the GOP held a lead that was on average 5.1 percentage points larger in the landline sample than in the combined landline and cell phone sample.
In six polls conducted in the fall of 2008, Barack Obama’s lead over John McCain was on average 2.4 percentage points smaller in the landline samples than in the combined samples.
In Pew Research’s final pre-election poll in 2010, the landline sample of likely voters found Republican candidates ahead 51%-39%, a 12-point lead. In the sample that combined landline and cell phone interviews, the Republican lead was 48%-42%, a six-point advantage. The national vote for House candidates is not yet final; currently, Republicans lead by approximately a seven-point margin.
The key comparison in this analysis is between estimates based on the landline sample alone, weighted to a standard set of demographic characteristics, and the combined sample of landline and cell phone interviews, weighted to the same demographic characteristics plus telephone status and usage (see the methodology statement for more details).
A previous analysis in May found a growing potential bias in landline only surveys (See “Assessing the Cell Phone Challenge,” May 20, 2010). Previous Pew Research studies on this subject had generally found only small and often non-significant differences between landline and dual frame samples (May 2006, June 2007, Jan. 2008, Dec. 2008).
The new analysis also finds consistent differences between the landline and combined samples when comparing registered voters. In five of six Pew Research polls conducted during 2010, there was a three to six percentage point difference on the margin among registered voters. Only one poll showed no difference.
Cell Respondents Less Republican, More Democratic
The difference in estimates produced by landline and dual frame samples is a consequence not only of the inclusion of the cell phone-only voters who are missed by landline surveys, but also of those with both landline and cell phones — so called dual users — who are reached by cell phone. Dual users reached on their cell phone differ demographically and attitudinally from those reached on their landline phone. They are younger, more likely to be black or Hispanic, less likely to be college graduates, less conservative and more Democratic in their vote preference than dual users reached by landline.
Among dual users reached by landline, Republicans had a 12-point advantage among likely voters. But the GOP lead was only five points among dual users reached by cell phone. Among cell phone only voters, there was no Republican lead (a nominal 47%-44% Democratic edge). These patterns were, if anything, stronger among the broader sample of registered voters than among likely voters.
Duals reached by landline made up a smaller proportion of voters in the dual frame sample than the landline sample because of the addition of the cell only respondents and the duals reached by cell. The latter two groups comprised about a third of all voters in the dual frame sample.
While dual users reached by landline were the majority of voters in both the dual frame sample and the landline sample, they were a far smaller proportion of voters in the dual frame sample because of the addition of the cell only respondents and the duals reached by cell. The latter two groups comprised about a third of all voters in the dual frame sample.1
Those who only have a landline also were somewhat more Democratic in their vote preference than respondents reached on their landline who also have a cell phone. However, in both the landline sample and the combined sample, landline only respondents made up only about 10% of the sample.2
Further evidence of the impact of dual users reached by cell phone can be seen in a comparison of blended samples that include landline and cell phone interviews with those that include landline and just the cell-only respondents. Adding just the cell-only respondents to the landline samples reduced the GOP advantage, but not as much as including all of the interviews from the cell phone sample.
Republicans led the Democrats by average margin of 9.5 percentage points among likely voters when cell-only respondents were included with the landline survey. The Republican advantage was 7.6 percentage points in the combined sample that includes all of the interviews completed by cell.
Young Voters and Cell Phones
Cell phones pose a particular challenge for getting accurate estimates of young people’s vote preferences and related political opinions and behavior. Young people are difficult to reach by landline phone, both because many have no landline and because of their lifestyles. In Pew Research Center surveys this year about twice as many interviews with people younger than age 30 are conducted by cell phone than by landline, despite the fact that Pew Research samples include twice as many landlines as cell phones.
According to the latest estimates from the National Center for Health Statistics, in the second half of 2009, 38% of 18 to 24 year olds and 49% of 25 to 29 year olds lived in households that had no landline. And research has shown that people younger than age 30 who are cell phone only can have different behaviors and attitudes than those who are reachable by a landline phone.3
Estimates from the combined landline and cell sample based on the last three pre-election Pew Research surveys showed Democrats with a 53%-to-38% lead over Republicans among registered voters younger than age 30. But estimates based only on interviews from the landline sample showed Democratic and Republican candidates running about even among young voters — 49% said that if the elections were held today they would vote for the Democratic candidate, while 45% backed the Republican candidate in their district. The difference in the margin between the combined sample and the landline sample was 11 points.
These sample differences in voter preference were not limited just to voters younger than age 30. Estimates based on the landline and cell interviews produced slightly less support for Republican candidates and greater support for Democratic candidates in voters ages 30-39 and 40-49. Among registered voters in both of these age groups, there was a difference in the margin of five percentage points. Among registered voters ages 50 and older, there was little difference in the margin between the combined sample and the landline samples.
Comparing the vote preference of likely voters across age groups produces a similar result, although the estimates are not quite as robust because of the smaller sample sizes. Because young people are significantly less likely than older adults to be classified as likely voters, the sample size of young likely voters is reduced considerably.
Characteristics of Dual Users Reached by Cell and Landline
Dual users reached by cell phone are much younger than those reached by landline — 22% of duals interviewed on their cell phone are younger than age 30, compared with only 8% of duals reached on their landline.
The cell phone only are younger than dual users and those with only a landline; 42% of people with only a cell phone are younger than age 30.
Half of those who have only a landline phone are ages 65 or older — far higher than in any other phone use category.
There also are differences by education. More of the duals reached by cell than by landline have only a high school education or less (33% vs. 29%) while more duals reached on their landline are college graduates (43% vs. 38%).
Seven-in-ten (70%) duals interviewed on their cell phone are non-Hispanic whites, compared with 79% of duals reached on their landline.
Hispanics and African-Americans comprise a larger proportion of duals reached on their cell phone than on their landline phone.
As the voter preferences would suggest, duals reached on their cell phone are somewhat more Democratic in their party affiliation than those interviewed on their landline phones.
By a 47%-to-43% margin duals reached on their cell phone identify with the Democratic Party or lean Democratic.
This balance is nearly reversed among duals reached on their landline — 46% are Republicans or lean Republican, while 43% are Democrats or lean Democratic.
Both the cell onlys and those with only a landline phone are more Democratic than Republican in their party identification.
Duals interviewed on their landline phone are somewhat more conservative in their political views than those reached by cell (43% vs. 40%).
Cell onlys, by contrast, are more liberal than duals reached by landline or cell phone.
Duals reached by cell also are less likely to be registered to vote than those interviewed on their landline phone (80% vs. 88%).
Voter registration is lowest among those with only a cell phone — just 60% are registered voters.
ABOUT THE DATA
Most results in this report are from six Pew Research Center surveys conducted in 2010. All six surveys were based on telephone interviews conducted among a nationwide samples of adults, 18 years of age or older, living in the continental United States. The table below shows the field dates and sample sizes for registered and likely voters for each survey.
The late October survey was conducted under the direction of Princeton Survey Research Associates International with data collection by Princeton Data Source and Abt/SRBI Inc. The mid-October, September, August and March surveys were conducted under the direction of Princeton Survey Research Associates International with data collection by Princeton Data Source. The June survey was conducted by Abt SRBI, Inc. Interviews were conducted in English only for the mid-October and late October surveys. Interviews were conducted in English and Spanish for the other four surveys. The landline and cell phone samples for all six surveys were provided by Survey Sampling International.
The combined landline and cell phone sample are weighted using an iterative technique that matches gender, age, education, race/ethnicity, region, and population density to parameters from the March 2009 Census Bureau’s Current Population Survey. The sample also is weighted to match current patterns of telephone status and relative usage of landline and cell phones (for those with both), based on extrapolations from the 2009 National Health Interview Survey.
The weighting procedure also accounts for the fact that respondents with both landline and cell phones have a greater probability of being included in the sample and adjusts for household size within the landline sample. The landline sample is weighted using the same techniques as the combined sample, except that the telephone status and usage parameter is excluded, and there is no correction for respondents with both landline and cell phones, since they have only one chance of being included in the sample. The landline plus cell only sample is weighted using the same techniques as the combined sample, except that there is no correction for respondents with both landline and cell phones, since they have only one chance of being included in the sample.
The significance tests comparing the estimates based on the combined landline and cell sample with those from the landline sample account for the overlap in the two samples since the landline respondents are included in both samples. The tests were conducted using software that accommodates complex survey samples.
Sampling errors and statistical tests of significance take into account the effect of weighting. The following table shows the error attributable to sampling that would be expected at the 95% level of confidence for the combined landline and cell sample and the landline sample only for registered and likely voters:
In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.
Likely voter estimates are based on a seven-item turnout scale that includes the following questions: thought given to the election, have ever voted in your precinct or election district, how often you follow government and public affairs, frequency of voting, whether you plan to vote in the election, chance of voting in the election on an 10-point scale, and whether you voted in the 2008 election. Those in the mid-October and late October surveys who said they already voted were automatically counted as likely voters. More details about the Pew Research Center’s methodology for estimating likelihood to vote are available at: http://pewresearch.org/pewresearch-org/politics/methodology/files/UnderstandingLikelyVoters.pdf.
1. The ratio of dual users reached by landline vs. those reached by cell phone is dependent on the overall ratio of landline to cell interviews. In Pew Research Center surveys conducted this year, a third of the completed interviews were conducted by cell phone and two-thirds were completed by landline.
2. The ratio of dual users reached by landline vs. those reached by cell phone is dependent on the overall ratio of landline to cell interviews. In Pew Research Center surveys conducted this year, a third of the completed interviews were conducted by cell phone and two-thirds were completed by landline.
3. Stephen J. Blumberg and Julian V. Luke. Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults. 2007 Public Opinion Quarterly 2007 71: 734-749. Avalable at http://poq.oxfordjournals.org/cgi/reprint/71/5/734.
