Public opinion pollsters in the United States survey adults of every racial and ethnic group. Historically, though, while the attitudes of white, Hispanic and Black adults are readily reported, those of Asian Americans are not. That raises a question: If Asian Americans are part of the survey sample, why don’t analysts report their views in the same way?
The answer, at least partly, comes down to math.
Asian Americans make up 5.9% percent of the U.S. adult population. That means that a perfectly representative national survey of 1,000 adults would include 59 Asian Americans – far too few to support reliable estimates. Pew Research Center is committed to surveying people of all demographic backgrounds, but we only feel comfortable reporting the views of Asian Americans and other smaller groups when we are confident that the data paints an accurate picture.
Consider a hypothetical poll of 1,000 adults, in which half of the surveyed Asian Americans favor a particular policy. In this theoretical survey, the margin of sampling error for Asian Americans would be at least 13 percentage points, which means the actual figure might be as low as 37% or as high as 63%. This result is simply too imprecise to report with confidence.
Fortunately, an important development in survey research – the migration of polling from the phone to the internet – is making it more feasible for some pollsters, including Pew Research Center, to publish estimates for Asian Americans. This trend won’t change the industry overnight, but it is already leading to a small uptick in the publication of Asian American estimates.
As more polls are done online, they are sampling from large groups of respondents, or “survey panels.” When working with a large panel, researchers can have access to reliable sample sizes for less populous groups, including Asian Americans. Pew Research Center and other organizations with large national panels are currently able to release estimates for Asian Americans in at least some of their surveys.
While this data stands to increase the visibility of Asian Americans in polling, there is an important caveat: Typically, Asian American estimates from online survey panels are based just on English (or Spanish) speakers. Those who speak any of the dozens of common Asian languages and are not proficient in English or Spanish generally cannot participate. That matters because English-speaking Asian Americans differ in important ways from those who don’t speak English, including in their income and education levels and their views on social issues.
The infusion of new online survey data is clearly an improvement on the past, when Asian American figures were virtually nonexistent in public opinion surveys of U.S. adults. But these estimates still offer a limited view of the entire Asian American population. The analysis below looks in greater detail at recent developments – and ongoing challenges – in surveying Asian Americans.
A major shift in how some pollsters recruit respondents
The chronic problem of having too few Asian American respondents stems from a period when pollsters surveyed a fresh sample of Americans with each new poll. A pollster might interview 1,000 people one month, for example, and a different 1,000 people the next month. That approach, known as cross-sectional sampling, was common in telephone polling from roughly 1980 to 2010. Cross-sectional samples are generally representative of the population, but one clear downside is that they make it difficult for analysts to report the views of smaller subgroups with accuracy (even though members of these groups are included in the overall survey results).
The recent increase in online survey work has helped address this problem, at least to an extent, by polling much larger groups of people and doing so more than once.
At Pew Research Center, we randomly recruit respondents offline and give them instructions for taking our surveys online (as some other pollsters do, too) through our American Trends Panel. This offline recruitment – centered around sampling home addresses and sending mailings with a small amount of money inside to incentivize participation – is time- and resource-intensive. As a result, we ask each participant to take multiple surveys, not just one. This approach, called a survey panel, is gradually supplanting many cross-sectional surveys throughout the industry. It also has a clear benefit when it comes to polling Asian Americans: By building the size of our panel over time, we can add more Asian American respondents without having to start from scratch with each survey.
Not all survey panels are recruited offline. In fact, most are recruited with inexpensive convenience samples of internet users. That has implications for data quality, but the basic principle of a panel is the same – recruit a large number of people to take surveys on an ongoing basis (for example, one per month for several years).
Some surveys are reporting Asian American estimates
One upshot of the rise in survey panels is that many of them are large enough to include hundreds or even thousands of Asian American adults. Our American Trends Panel, for instance, has roughly 15,000 U.S. adults, about 400 of whom self-identify as Asian American.
Many of our surveys interview only a random fraction of these panelists (e.g., 2,500 interviews). In these cases, the Asian American sample is too small to support reliable estimates. But when feasible, we attempt to interview nearly every member of the panel, making it possible to report reliable estimates for Asian adults. This new Pew Research Center report is an example of one such survey, and we plan to release more in the future.
Other polling organizations also sometimes report the views of Asian Americans. The Understanding America Study at the University of Southern California (USC) contains about 8,500 U.S. households. A number of its reports contain a sufficient Asian American sample size to report estimates. Additional panels that have supported surveys producing Asian American estimates include Rand’s American Life Panel, NORC’s AmeriSpeak Panel and the Ipsos KnowledgePanel. In all cases, estimates reported are for English-speaking Asian American adults.
Still, this practice is far from universal. A typical poll (say, one based on 1,000 to 2,500 interviews) doesn’t include enough Asian Americans to report reliable data about their views. The only context in which that is common is state-level polling in California, where 16% of residents are Asian American. In that state, pollsters like the Public Policy Institute of California routinely publish reliable Asian American estimates.
Outside of the polling industry, a number of scholars have long used surveys to study Asian Americans. Perhaps the best-known source for such data is AAPIData.com, which features demographic data and policy research on Asian Americans and Pacific Islanders. In order to efficiently reach Asian Americans, that line of research has typically leveraged list-based samples, which identify potential Asian American respondents based on several characteristics.
Language and other diversity among Asian Americans
Panel surveys like those at USC or Pew Research Center are typically administered in English and Spanish. That works well for estimates for all U.S. adults, as 99% of U.S. adults speak one or both of those languages well enough to complete a survey. For Asian Americans, however, such surveys are limiting. Asian Americans are a heavily immigrant community: More than three-quarters of Asian American adults were born in another country, and about a third (32%) do not speak English “very well,” according to self-reported data collected in the Census Bureau’s American Community Survey.
Importantly, those who speak only Asian languages differ from their English-speaking counterparts in several ways. Non-English-speaking Asian Americans tend to be less wealthy, more socially conservative and have less formal education than those who do speak English. (We know this from past research: For example, a large custom survey of 3,511 Asian Americans conducted for the Center in 2012 found that 60% of those interviewed in English said abortion should be legal in most or all cases, compared with 35% of those interviewed in one of the seven Asian languages offered: Cantonese, Hindi, Japanese, Korean, Mandarin, Tagalog and Vietnamese.) These differences can have implications for survey research. For example, it may be problematic to rely on an English-speaking sample of Asian Americans in a survey that addresses a subject like religious orthodoxy or the immigrant experience.
But the diversity of Asian Americans extends beyond language. They also hail from many different cultural and ethnic backgrounds across the vast continent of Asia. This diversity, too, has implications when it comes to survey work.
For example, when reported as a whole, the views of Asian Americans naturally represent some groups more than others. The views of Chinese, Indian and Filipino Americans are more heavily represented than those of Pakistani or Laotian Americans because the former groups make up a much larger share of the Asian American population than the latter groups. Critically, the more populous groups tend to have higher income levels and more formal education than the less populous Asian origin groups.
Overall, the increasing use of online panels makes it more feasible for some polling organizations to report Asian American estimates, but a number of caveats clearly remain. It’s important for poll consumers to bear in mind that the picture they might be seeing of Asian Americans is an incomplete one.