At Pew Research Center, we work hard to make sure our surveys include respondents who are as diverse as the American public. When it comes to religious affiliation, our surveys include respondents who identify with dozens of groups – not just large ones such as Protestants, Catholics and people with no religion, but also smaller ones in the U.S. such as Buddhists, Muslims, Unitarians, Wiccans and others.
The Center measures respondents’ religious identity by asking, “What is your present religion, if any?” and then presenting the following options: Protestant, Roman Catholic, Mormon, Orthodox (such as Greek or Russian Orthodox), Jewish, Muslim, Buddhist, Hindu, atheist, agnostic, something else, or nothing in particular. Those who say “something else” are then prompted to describe their religion in their own words, and we code it as accurately as possible. This is how we know we have a wide range of religious faiths represented in our surveys.
Still, many Center publications report the views only of Protestants, Catholics and religiously unaffiliated Americans, along with the two or three largest subgroups in each – such as evangelical Protestants, Hispanic Catholics and atheists. Missing from these reports are the opinions of members of non-Christian religions, as well as smaller Christian groups such as Orthodox Christians, Episcopalians and members of the Church of Jesus Christ of Latter-day Saints (sometimes called Mormons).
Since we often field questions about why we do not include more religious groups in our charts and tables, we conducted this analysis to help offer an explanation. We looked at a recent survey that contacted all of the members of the American Trends Panel (ATP), where Pew Research Center now conducts most of its surveys of the U.S. public. The survey analyzed here was conducted May 16-22, 2022, among 10,188 respondents.
The ATP is an online survey panel that is recruited through national, random sampling of residential addresses. Respondents who are invited but do not have internet access are provided with an internet-connected tablet which they can use to take the surveys online. This way, nearly all U.S. adults have a chance of selection. The survey is weighted to be representative of the U.S. adult population by gender, race, ethnicity, partisan affiliation, education and other categories. Read more about the ATP’s methodology.
This is because Protestants, Catholics and religious “nones” – including atheists, agnostics and those who say their religion is “nothing in particular” – together make up roughly 90% of the U.S. population. With no other religious group making up more than 2% of the population, our American Trends Panel (ATP) – a group of more than 10,000 adults who take our U.S. surveys online – does not have enough respondents in these smaller groups to give us confidence that they are representative of all of their members across the country.
The general policy of Pew Research Center is to restrict most of our reporting to groups that are large enough to have an effective sample size of at least 100 people. Effective sample size is a statistical term that refers to the number of interviews we conduct as part of a survey, adjusted for the loss in precision associated with weighting – the standard adjustments made to the data to ensure it is representative of the overall U.S. population. Generally, groups that have an effective sample size of less than 100 will have a margin of error of more than 10 percentage points, leaving a wide range of uncertainty about these groups’ true opinions. As a result, we think it is better not to report these figures at all rather than to potentially give readers a false sense of certainty about our data.
We occasionally do report findings for groups when the effective sample size is less than 100 interviews. When we do this, we also publish a confidence interval for each data point as a way of showing readers that survey estimates for the group are less precise than most of the data we report. But even this technique has some strict limits. If a group has fewer than 100 raw interviews or an effective sample size of less than 50, we will not publish the estimates at all – not even with a confidence interval. With an effective sample size of fewer than 50 respondents, the margin of error is often over 20 percentage points. In other words, if 55% of the survey respondents in such a group hold a certain view, it is possible that the view is held by as few as 35% or as many as 75% of the members of the group across the country.
Based on these guidelines, we can consistently analyze the largest Christian traditions, as well as religiously unaffiliated people, but we can rarely report on members of non-Christian faiths – even though the ATP has a relatively large overall sample size compared with most telephone surveys we conducted in the past. While the typical ATP survey has roughly 10,000 respondents overall, it includes just over 50 Muslims and Hindus each, for example. Standard weighting procedures further diminish the statistical power of these samples.
In terms of their share of the overall population, the largest non-Christian religious group in the U.S. is Jews, who account for 1.7% of U.S. adults, according to our most recent estimate. Other groups, such as Muslims and Hindus, are smaller still – and some groups also are less likely than others to respond to surveys.
When taken together, members of non-Christian faiths make up about 5% of the respondents in our typical ATP surveys. It would be possible to show responses from an aggregate “Other faiths” category, but this would require combining groups as diverse as Hindus, Unitarians and Yoruba into a single category, which in our view would rarely be informative.
The same issues limit our ability to report on smaller Christian groups. For example, the ATP also has a relatively small effective sample size of Jehovah’s Witnesses. And while we can generally subdivide Protestants into those who identify with evangelical, mainline or historically Black denominations, we cannot report on most specific denominations. Groups ranging from Seventh-day Adventists to members of the African Methodist Episcopal (AME) Church are too small to report on separately in most of our reports.
In order to understand the views, beliefs and practices of smaller religious groups, we instead do much larger studies. Sometimes these are custom studies that contact tens of thousands of people in order to draw a large enough sample of a specific group, such as our surveys of U.S. Jews, Muslims or Mormons. And on two occasions, Pew Research Center has done a large-scale survey with over 35,000 interviews in order to understand the overall American religious landscape. Projects of this type make it possible not only to speak about small religious groups in general terms, but also to unpack their rich internal diversity.
In addition, the Center often reevaluates how it can best use its resources to provide the highest-quality data and analysis we can. The ATP has grown in its size and capabilities over the years, and we are hopeful that more routine analysis of smaller religious groups may become possible in the future.
Read also: Polling methods are changing, but reporting the views of Asian Americans remains a challenge