International Survey Research
Survey mode and sample design
Pew Research Center’s cross-national studies are fielded in a range of polling environments, from countries where telephone surveys are regularly administered to nations where face-to-face surveys are the only means of achieving reliable, nationally representative samples. In all cases, the center adopts the survey mode that follows local best practices and meets Pew Research standards for methodological rigor. As of 2014, a majority of the overseas polls conducted by Pew Research Center are administered face-to-face, with telephone surveys generally limited to North America, Western Europe and select countries in the Asia-Pacific region, such as Australia, Japan and South Korea. All cross-national studies are based on interviewer-administered survey instruments.
Pew Research Center constantly seeks to improve and refine its overseas survey methodology, whether the mode is face-to-face or telephone. The center, for example, is intensifying its efforts to collect robust para-data (information about the circumstances and conduct of a survey). In addition, the center continues to monitor methodological developments, including the growing role of handheld devices and computers in assisting face-to-face interviews, delivery of questions via text messages (SMS) and online panels. At this time, Pew Research has decided not to employ opt-in, nonprobability panels as a means of measuring overseas public opinion, due to concerns about the representativeness of such surveys and the lack of an established metric for gauging the accuracy of such polls.
Pew Research Center regularly conducts telephone surveys in countries where such polls are an established feature of opinion research and telephone penetration (landline and cell) includes 90% or more of the total population. Examples of such countries include: Canada, the United Kingdom, France, Germany, Spain, the Czech Republic, Australia, Japan and South Korea. As in the United States, the number of people who own a cellphone has skyrocketed in many of these countries over the past decade. However, in some countries, large segments of the population (more than 70%) continue to own and use landline phones. Pew Research designs its telephone surveys so that the distribution of landline and cellphone users, not simply owners, is accurately reflected in each country-specific sample. The center tries to stay abreast of shifts in the balance between landline and cellphone users, especially as the proportion of people who rely solely or mostly on cellphones for communication increases over time.
Pew Research relies on random digit dialing (RDD) for all its overseas telephone surveys (for more on RDD methodology, see random digit dialing). In conjunction with RDD, the center employs two types of samples, depending on current phone-use patterns and available sampling frames (i.e., lists) of landline and cellphone numbers in a given country: combined cell and landline samples and cell-only samples.
Combined cell and landline samples are typical of Pew Research Center surveys in the UK, France, Germany and Spain. In each case, the samples are split between respondents who have access to only a landline phone, only a cellphone or both types of phones. In two countries included in the center’s cross-national studies – the Czech Republic and South Korea – high cellphone ownership rates (90% or more), combined with people’s relatively high propensity to respond to surveys on their cellphones, have led to a decision to shift to cellphone-only samples. These surveys entail the random sampling of all registered cellphone exchanges, as opposed to reliance on sampling frames acquired from individual cellphone carriers.
Pew Research Center continues to rely on face-to-face surveys as its primary mode of data collection in countries extending from Latin America, to Eastern Europe, to Africa and Asia. In almost all cases, random samples are based on multi-stage, cluster designs. What this means is that rather than randomly selecting individuals directly (by phone, for example), we first randomly select clusters of individuals – beginning with relatively large territorial units, akin to states or counties in the U.S. Once these primary clusters are selected, we randomly select smaller territorial units, until we work our way down to city blocks or villages. At this stage, interviewers either visit addresses selected randomly from a list, or they follow a so-called “random walk” in which they visit every third or fourth residence along a set route. At each residence, interviewers randomly select a respondent by using a Kish grid (a detailed list of all household members) or by selecting the adult who has had the most recent birthday.