--- title: "How well synthetic polls capture the diversity of public opinion" description: "AI survey respondents are more likely than humans to agree on one answer choice or avoid a response option entirely." date: "2026-09-30" authors: - name: "Athena Chapekis" job_title: "Computational Social Scientist" link: "https://www.pewresearch.org/staff/athena-chapekis/" - name: "Arnold Lau" job_title: "Research Methodologist" link: "https://www.pewresearch.org/staff/arnold-lau/" - name: "Samuel Bestvater" job_title: "Associate Director, Data Labs" link: "https://www.pewresearch.org/staff/samuel-bestvater/" - name: "Sono Shah" job_title: "Former Associate Director, Research" link: "https://www.pewresearch.org/staff/sono-shah/" - name: "Andrew Mercer" job_title: "Principal Methodologist" link: "https://www.pewresearch.org/staff/andrew-mercer/" - name: "Aaron Smith" job_title: "Director, Data Labs" link: "https://www.pewresearch.org/staff/aaron-smith/" url: "https://www.pewresearch.org/data-labs/2026/09/30/how-well-synthetic-polls-capture-the-diversity-of-public-opinion/" categories: - "Artificial Intelligence" - "Methodological Research" - "Uncategorized" --- # How well synthetic polls capture the diversity of public opinion **About this research** This Pew Research Center report examines whether AI-generated survey data can accurately duplicate the results of high-quality public opinion polls. We conducted this methodological experiment to better understand an emerging method in survey research. The Center believes that speaking to the public is essential to measuring public opinion and has no current or future plans to use AI models to generate survey results. Read more about our [AI policy](https://www.pewresearch.org/decoded/2026/07/30/how-pew-research-center-is-and-is-not-using-ai-in-our-work-2/). For this study, we had an AI model take on the role of real respondents from our [American Trends Panel](https://www.pewresearch.org/the-american-trends-panel/) (ATP) and answer the same surveys administered to human panelists. These surveys included many different question formats and asked about a wide range of topics, attitudes and behaviors. #### Why did we do this? The Center regularly studies and evaluates [advances and trends in polling research](https://www.pewresearch.org/topic/methodological-research/survey-methods/). As AI-based polling becomes [more widely used in the industry](https://www.nytimes.com/2026/04/06/opinion/ai-polling.html), we wanted to understand the data these “synthetic samples” produce and how it compares with high-quality surveys of real humans on topics of broad public interest. That question is the focus of this study; it does not address other uses of AI models in polling. #### How did we do this? We used a “digital twins” approach for this study, asking an AI model to adopt the personas of real humans who are members of the ATP. We gave the model a [wide range of information about each panelist](#_Conditioning_information), including their self-reported demographic information and their responses to questions on our [2025 political typology survey](https://www.pewresearch.org/politics/2026/06/10/beyond-red-vs-blue-the-political-typology/). We then showed the model questions from three different ATP surveys conducted in the first half of 2026 and asked it to respond to those same questions as the assigned persona. The model was given each question on the survey in order from start to finish, with exactly the same instructions and survey programming that the real humans who took it received. To make a direct comparison between our AI and human polls, we only replicated surveys for the ATP panelists from each wave who had also completed the political typology survey. As a result, figures for U.S. adults in this report may differ slightly from those previously published by the Center. We tested multiple models, but all synthetic results unless noted otherwise come from Anthropic’s Claude Opus 4.6. At the time we conducted this study, Opus 4.6 was the most recent Anthropic model available for commercial use and had the best performance of several we evaluated. Here are the [survey questions from Wave 185](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/01/PP_2026-01-29_views-of-trump_questionnaire.pdf), [Wave 190](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/04/pg_2026.04.28_us-role-world_questionnaire.pdf) and [Wave 192](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/05/PP_2026.5.11_national-problems_questionnaire.pdf) used for this analysis, the [detailed responses](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/09/pl_2026.09.30_silicon-samples_topline.pdf) from the ATP and AI surveys, and the [methodologies](https://www.pewresearch.org/data-labs/2026/09/30/methodology-silicon-samples/) for the original ATP survey waves and our AI surveys. Traditional public opinion polls are often marked by the diverse views they show within a population. While the broad contours of public sentiment are often clear, there are very few opinions that are universally held. By the same token, most of the answer choices on a typical survey question – from the extreme to the middle-of-the-road – are selected by at least a modest number of respondents. In our analysis of synthetic polling, AI respondents tended to eliminate a great deal of this diversity. On some questions, the model assumed most Americans’ views are far more extreme than they actually are. On others, it assigned the vast majority of respondents to the middle category in a set of choices. And across questions of all types, our AI survey-takers collectively avoided certain answer choices in ways that have no parallel among their human counterparts. Even if the model’s results point in the same general direction as true public opinion, they ultimately paint a picture that is less diverse and more one-note than what we see in actual polls of human respondents. *This analysis is part of a larger evaluation of AI-generated synthetic samples in public opinion research. Read* *[a summary of the main findings](https://www.pewresearch.org/data-labs/2026/09/30/can-ai-stand-in-for-human-survey-takers-not-really/)* *and refer to the* *[methodology](https://www.pewresearch.org/data-labs/2026/09/30/methodology-silicon-samples/)* *for more details on how we conducted our synthetic poll and compared it with real survey results.* ### How synthetic polls represent attitudes and opinions On numerous questions in our surveys, our synthetic poll gave the appearance of a public that is far more lockstep – and often far more certain – in its attitudes than it actually is. A question about whether the United States or China benefits more from trade between the two countries is a good example. Our synthetic poll estimated that fully 98% of Americans think China benefits more from that relationship. That’s 56 percentage points higher than the real share of U.S. adults who feel this way. It also obscures notable shares who feel that the U.S. benefits more (10%), who say both countries benefit equally (24%), and who say they aren’t sure (19%). And on a series of questions about the changing influence of different countries on the global stage, the model tended to gravitate toward certain answer choices for no apparent reason. Our AI poll estimated that: - 88% of Americans think India has been getting stronger in recent years (while the actual share is 30%). - 77% say Canada has been getting weaker (actually 23%). - 100% say China has been getting stronger (actually 62%). - 92% say Iran’s influence has stayed the same (actually 33%). ### Synthetic respondents differ from real Americans on descriptors of Trump *% who say each describes Donald Trump …* | Item | Response | U.S. adults | Synthetic respondents | | --- | --- | --- | --- | | A good role model | Very well | 9 | 0 | | A good role model | Fairly well | 17 | 9 | | A good role model | Not too well | 20 | 42 | | A good role model | Not at all well | 53 | 49 | | Keeps his promises | Very well | 16 | 5 | | Keeps his promises | Fairly well | 22 | 48 | | Keeps his promises | Not too well | 21 | 44 | | Keeps his promises | Not at all well | 40 | 3 | | Stands up for what he believes in | Very well | 38 | 39 | | Stands up for what he believes in | Fairly well | 27 | 61 | | Stands up for what he believes in | Not too well | 14 | 0 | | Stands up for what he believes in | Not at all well | 21 | 0 | Note: Figures for U.S. adults may differ slightly from those previously published by the Center because they are based on a subset of respondents to the original survey. Source: Survey of U.S. adults and “digital twins” synthetic analysis using Claude Opus 4.6 set to low reasoning with extended profile information and expert reflection. Survey was conducted April 20-26, 2026 (replicated April 27-May 1).“Can AI Stand In for Human Survey-Takers? Not Really” Broadly speaking, our model had a tendency to steer away from choices at the end of scales (like “very well” or “not at all well”) and toward middle choices like “somewhat.” On a series of questions about President Donald Trump’s personal characteristics, 0% of our AI respondents said that phrases like “honest,” “mentally sharp” or “cares about the needs of ordinary people” describe him very well. In reality, anywhere from one-in-ten to one-in-four Americans say these describe the president very well. At the same time, this trend was not always consistent across questions – or even within the same set of questions. When presented with a list of 13 different issues facing the country, a substantially *larger* share of AI respondents than human panelists described several as a “very big problem.” There were only four issues for which the more extreme option was chosen by our human panelists more often. ### How synthetic polls represent reported behaviors AI-generated survey responses also fail to capture the full diversity of experiences humans report about their own lives. This can be seen clearly in a series of questions we asked about Americans’ sleep and travel habits. ### On questions about sleep, AI surveys lump responses into middle categories | Hours of sleep | 4 hours of sleep or less | 5 to 6 hours of sleep | 7 to 9 hours of sleep | 10 hours of sleep or more | | | --- | --- | --- | --- | --- | --- | | U.S. adults | 5 | 39 | 54 | 2 | | | Synthetic respondents | 0 | 69 | 31 | 0 | | | | | | | | | | U.S. adults | 6 | 14 | 36 | 33 | 10 | | Synthetic respondents | 0 | 1 | 77 | 21 | 0 | Note: Figures for U.S. adults may differ slightly from those previously published by the Center because they are based on a subset of respondents to the original survey. Source: Survey of U.S. adults and “digital twins” synthetic analysis using Claude Opus 4.6 set to low reasoning with extended profile information and expert reflection. Survey was conducted April 20-26, 2026 (replicated April 27-May 1).“Can AI Stand In for Human Survey-Takers? Not Really” On a question about **how often people have trouble falling asleep,** fully 98% of synthetic respondents say they experience this some days or rarely. Although a majority of Americans choose one of these two options, 30% in total say this happens to them every day, most days or never. The synthetic poll also lumps the vast majority of Americans into the “some days” category on a similar question about how often they have trouble staying asleep. When it comes to **how much sleep they get per night,** the synthetic poll overstated the share of Americans who get five to six hours of sleep per night while understating the share who get seven to nine hours of sleep. The synthetic poll also estimated that *no* Americans get very little (four hours or less) or quite a bit (10 hours or more) of sleep in a night – but in reality, nearly one-in-ten Americans fall into one of these categories. And on a question about **the number of** **countries people have visited outside the United States,** the synthetic poll was fairly close to the actual population on most response options. But it estimated that just 1% of Americans have visited 10 or more countries. In reality, 13% of U.S. adults say that they have done this. ### Synthetic respondents often avoid certain answer choices entirely On a typical Pew Research Center survey, it’s rare for a single answer choice on a given question to get zero responses. This did not happen on any of the questions across the three original survey waves we evaluated for this study. These “zero selection” response options occurred much more often in our AI survey. Fully 47% of questions in our synthetic poll had at least one answer choice that was not selected by a single synthetic respondent. This disparity between the human and synthetic respondents is also visible at other thresholds: - Just 1% of questions in our human poll had a response option that was selected by fewer than 1% of respondents, versus 66% of questions on the synthetic poll. - 27% of questions in the human poll had a response option selected by fewer than 5% of respondents, versus 77% of questions on the synthetic poll. --- **Next:** [How synthetic polling results change based on the AI model](https://www.pewresearch.org/data-labs/2026/09/30/how-synthetic-polling-results-change-based-on-the-ai-model.md)