---
title: "Synthetic surveys and ‘timely and topical’ questions"
description: "Synthetic surveys fail to replicate public opinion on timely and topical issues in early 2026, including presidential approval and views of data centers."
date: "2026-09-30"
authors:
  - name: "Athena Chapekis"
    job_title: "Computational Social Scientist"
    link: "https://www.pewresearch.org/staff/athena-chapekis/"
  - name: "Arnold Lau"
    job_title: "Research Methodologist"
    link: "https://www.pewresearch.org/staff/arnold-lau/"
  - name: "Samuel Bestvater"
    job_title: "Associate Director, Data Labs"
    link: "https://www.pewresearch.org/staff/samuel-bestvater/"
  - name: "Sono Shah"
    job_title: "Former Associate Director, Research"
    link: "https://www.pewresearch.org/staff/sono-shah/"
  - name: "Andrew Mercer"
    job_title: "Principal Methodologist"
    link: "https://www.pewresearch.org/staff/andrew-mercer/"
  - name: "Aaron Smith"
    job_title: "Director, Data Labs"
    link: "https://www.pewresearch.org/staff/aaron-smith/"
url: "https://www.pewresearch.org/data-labs/2026/09/30/synthetic-surveys-and-timely-and-topical-questions/"
categories:
  - "Artificial Intelligence"
  - "Methodological Research"
  - "Uncategorized"
---

# Synthetic surveys and ‘timely and topical’ questions

**About this research**

This Pew Research Center report examines whether AI-generated survey data can accurately duplicate the results of high-quality public opinion polls.

We conducted this methodological experiment to better understand an emerging method in survey research. The Center believes that speaking to the public is essential to measuring public opinion and has no current or future plans to use AI models to generate survey results. Read more about our [AI policy](https://www.pewresearch.org/decoded/2026/07/30/how-pew-research-center-is-and-is-not-using-ai-in-our-work-2/).

For this study, we had an AI model take on the role of real respondents from our [American Trends Panel](https://www.pewresearch.org/the-american-trends-panel/) (ATP) and answer the same surveys administered to human panelists. These surveys included many different question formats and asked about a wide range of topics, attitudes and behaviors.

#### Why did we do this?

The Center regularly studies and evaluates [advances and trends in polling research](https://www.pewresearch.org/topic/methodological-research/survey-methods/). As AI-based polling becomes [more widely used in the industry](https://www.nytimes.com/2026/04/06/opinion/ai-polling.html), we wanted to understand the data these “synthetic samples” produce and how it compares with high-quality surveys of real humans on topics of broad public interest. That question is the focus of this study; it does not address other uses of AI models in polling.

#### How did we do this?

We used a “digital twins” approach for this study, asking an AI model to adopt the personas of real humans who are members of the ATP. We gave the model a [wide range of information about each panelist](#_Conditioning_information), including their self-reported demographic information and their responses to questions on our [2025 political typology survey](https://www.pewresearch.org/politics/2026/06/10/beyond-red-vs-blue-the-political-typology/). We then showed the model questions from three different ATP surveys conducted in the first half of 2026 and asked it to respond to those same questions as the assigned persona. The model was given each question on the survey in order from start to finish, with exactly the same instructions and survey programming that the real humans who took it received.

To make a direct comparison between our AI and human polls, we only replicated surveys for the ATP panelists from each wave who had also completed the political typology survey. As a result, figures for U.S. adults in this report may differ slightly from those previously published by the Center.

We tested multiple models, but all synthetic results unless noted otherwise come from Anthropic’s Claude Opus 4.6. At the time we conducted this study, Opus 4.6 was the most recent Anthropic model available for commercial use and had the best performance of several we evaluated.

Here are the [survey questions from Wave 185](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/01/PP_2026-01-29_views-of-trump_questionnaire.pdf), [Wave 190](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/04/pg_2026.04.28_us-role-world_questionnaire.pdf) and [Wave 192](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/05/PP_2026.5.11_national-problems_questionnaire.pdf) used for this analysis, the [detailed responses](https://www.pewresearch.org/wp-content/uploads/sites/20/2026/09/pl_2026.09.30_silicon-samples_topline.pdf) from the ATP and AI surveys, and the [methodologies](https://www.pewresearch.org/data-labs/2026/09/30/methodology-silicon-samples/) for the original ATP survey waves and our AI surveys.

Public opinion often seems slow to change, but the opinions and attitudes of the public can sometimes shift quickly and dramatically in response to current events. This is part of what’s interesting and challenging about studying public opinion. Sometimes people change their minds relatively rapidly, as happened with [former President George W. Bush’s increased approval rating](https://www.pewresearch.org/politics/2008/12/18/bush-and-public-opinion/) in the wake of 9/11. At other times, segments of the population can switch places on issues, as happened with Democrats’ and Republicans’ [perceptions of the social media platform X (formerly Twitter)](https://www.pewresearch.org/short-reads/2025/06/05/republicans-and-democrats-on-x-differ-over-the-sites-politics-and-their-experiences/) after Elon Musk purchased the company.

One appeal of AI polling is its speed. At least in theory, these polls can be fielded nearly instantly without the need to assemble a representative sample of humans who are willing to take the survey. At the same time, other ways of inferring public opinion without directly measuring it through polling (like statistical modeling) can struggle to identify changes in sentiment when they happen quickly, such as in response to breaking news.

We found that polling with AI models is no different. In our analysis comparing surveys of AI respondents with surveys of real humans, there were numerous instances in which the model failed to accurately reflect public opinion in unique cultural or economic circumstances. That is especially – but not always – true when those circumstances developed after the model’s knowledge cutoff.[2. numoffset="2" LLMs all have a knowledge cutoff – essentially the date on which the model stopped ingesting new data and training actually began. The model we used for most of this analysis, Claude Opus 4.6, has a knowledge cutoff in [May 2025](https://www.anthropic.com/transparency).]

*This analysis is part of a larger evaluation of AI-generated synthetic samples in public opinion research. Read* *[a summary of the main findings](https://www.pewresearch.org/data-labs/2026/09/30/can-ai-stand-in-for-human-survey-takers-not-really/)* *and refer to the* *[methodology](https://www.pewresearch.org/data-labs/2026/09/30/methodology-silicon-samples/)* *for more details on how we conducted our synthetic poll and compared it with real survey results.*

### Using synthetic polls to reproduce ongoing trends in public opinion

Pew Research Center polled Americans on their approval of President Donald Trump using the [American Trends Panel](https://www.pewresearch.org/the-american-trends-panel/) (ATP) seven times between the beginning of 2025 and May 2026, including on two surveys that we replicated with synthetic respondents.

In January 2026, 37% of U.S. adults approved of Trump’s job performance. By April, when the next replicated survey with this question was fielded, that had fallen to 34%. But our synthetic samples estimated Trump’s approval at a steady 46% for both waves.

### Synthetic surveys greatly overstate presidential approval in 2026

*% who approve of the way Donald Trump is handling his job as president*

| Date | All U.S. adults | U.S. adults, Dem/Lean Dem | U.S. adults, Rep/Lean Rep | All synthetic respondents | Synthetic respondents, Dem/Lean Dem | Synthetic respondents, Rep/Lean Rep | Dotted line total | Dotted line rep | Dotted line dem |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 2025-01-20 | 47 | 10 | 84 | – | – | – | – | – | – |
| 2025-04-01 | 40 | 7 | 75 | – | – | – | – | – | – |
| 2025-06-15 | 41 | 7 | 80 | – | – | – | – | – | – |
| 2025-08-15 | 38 | 6 | 74 | – | – | – | – | – | – |
| 2025-10-01 | 40 | 6 | 78 | – | – | – | 40.0 | 78.0 | 6.0 |
| 2026-01-15 | 37 | 6 | 72 | 46 | 7 | 88 | 46.0 | 88.0 | 7.0 |
| 2026-04-15 | 34 | 5 | 69 | 46 | 8 | 88 | – | – | – |

Note: Figures for U.S. adults may differ slightly from those previously published by the Center because they are based on a subset of respondents to the original survey.

Source: Surveys of U.S. adults and “digital twins” synthetic analysis using Claude Opus 4.6 set to low reasoning with extended profile information and expert reflection. Surveys were conducted Jan. 20-26, 2026 (replicated March 9-12 and April 7-10), and April 20-26, 2026 (replicated April 27-May 1).“Can AI Stand In for Human Survey-Takers? Not Really”

The last time Trump’s approval was actually that high in our polling was at the beginning of his second term in February 2025, when 47% of U.S. adults said they approved of the job he was doing. Not only did the synthetic replication in January 2026 fail to capture the decline in approval that had occurred since Trump’s return to office, but when we repeated the exercise in April, it gave us the exact same topline number.

If we look at the partisan breakdown on presidential approval over time, the story has been remarkably consistent for Democrats and Democratic leaners; the synthetic survey replicated their approval numbers quite closely. The biggest shift in attitudes in our human surveys has been a decline among Republicans and Republican leaners; our AI poll missed the mark for this group by as much as 19 percentage points.

Presidential approval is an especially long-standing trend question that the AI poll overestimated by a considerable margin in this case. But on its own, the fact that a question is trendable doesn’t tell us very much about the accuracy we should expect from an AI replication. The three surveys we evaluated in this study contained more than 120 questions we had asked at least once in the past. Collectively, these trend questions had an average error rate of 12 percentage points, regardless of how much opinion had actually changed over the life of the trend. That’s the same as the overall average error across all three survey waves.

### Using synthetic polls to measure public opinion on timely and topical issues

In addition to measuring how the model does at replicating some long-standing trends, we also looked at some new questions that asked about contemporaneous events the model might not have context for in its training data. These serve as examples of how synthetic polls can fail to capture how people feel about an issue when current sentiments break with past precedent in some way.

### Synthetic surveys underestimate public views on immigration agents wearing masks while working

*% who say it’s acceptable for …*

| id | Statement | Group | U.S. adults | Synthetic respondents | Error |
| --- | --- | --- | --- | --- | --- |
| Immigration officers to wear face coverings All respondents | Immigration officers to wear face coverings while working | All respondents | 38 | 17 | -21 |
| Immigration officers to wear face coverings Dem/Lean Dem | Immigration officers to wear face coverings while working | Dem/Lean Dem | 11 | 1 | -10 |
| Immigration officers to wear face coverings Rep/Lean Rep | Immigration officers to wear face coverings while working | Rep/Lean Rep | 67 | 35 | -32 |
| People to film immigration officers All respondents | People to film immigration officers | All respondents | 74 | 86 | 12 |
| People to film immigration officers Dem/Lean Dem | People to film immigration officers | Dem/Lean Dem | 88 | 99 | 11 |
| People to film immigration officers Rep/Lean Rep | People to film immigration officers | Rep/Lean Rep | 60 | 72 | 12 |

Note: Figures for U.S. adults may differ slightly from those previously published by the Center because they are based on a subset of respondents to the original survey.

Source: Survey of U.S. adults and “digital twins” synthetic analysis using Claude Opus 4.6 set to low reasoning with extended profile information and expert reflection. Survey was conducted Jan. 20-26, 2026 (replicated March 9-12 and April 7-10).“Can AI Stand In for Human Survey-Takers? Not Really”

Here are some “timely and topical” issues where our model failed to capture the state of public sentiment.

#### Immigration enforcement

U.S. Immigration and Customs Enforcement’s actions, such as “[Operation Metro Surge](https://www.npr.org/2026/02/12/nx-s1-5712280/minnesota-ice-surge-ends)” in winter 2025-26, generated public controversy over a range of issues, including federal agents’ wearing of masks or other face coverings to hide their identities while working.

Overall, the synthetic sample greatly underestimated the share of Americans who say it’s acceptable for immigration officers to cover their faces. Among Republicans and Republican leaners, the difference was even starker: 35% of synthetic Republicans find it acceptable, but the actual share among Republicans is nearly double that (67%).

#### Military conflict in Iran

### Synthetic surveys overestimate Republican support for military conflict in Iran

*% who …*

| id | Statement | Group | U.S. adults | Synthetic respondents | Error |
| --- | --- | --- | --- | --- | --- |
| Approve of U.S. military strikes against Iran All respondents | Approve of U.S. military strikes against Iran | Total | 36 | 48 | 12 |
| Approve of U.S. military strikes against Iran Dem/Lean Dem | Approve of U.S. military strikes against Iran | Dem/Lean Dem | 9 | 9 | 0 |
| Approve of U.S. military strikes against Iran Rep/Lean Rep | Approve of U.S. military strikes against Iran | Rep/Lean Rep | 67 | 93 | 26 |
| Confident Trump can make good decisions on Iran policy All respondents | Confident Donald Trump can make good decisions on Iran policy | Total | 35 | 45 | 10 |
| Confident Trump can make good decisions on Iran policy Dem/Lean Dem | Confident Donald Trump can make good decisions on Iran policy | Dem/Lean Dem | 7 | 4 | -3 |
| Confident Trump can make good decisions on Iran policy Rep/Lean Rep | Confident Donald Trump can make good decisions on Iran policy | Rep/Lean Rep | 66 | 89 | 23 |
| Think the U.S. is doing enough to prevent civilian casualties All respondents | Think the U.S. is doing enough to prevent civilian casualties | Total | 41 | 51 | 10 |
| Think the U.S. is doing enough to prevent civilian casualties Dem/Lean Dem | Think the U.S. is doing enough to prevent civilian casualties | Dem/Lean Dem | 19 | 10 | -9 |
| Think the U.S. is doing enough to prevent civilian casualties Rep/Lean Rep | Think the U.S. is doing enough to prevent civilian casualties | Rep/Lean Rep | 65 | 94 | 29 |
| Think the conflict will make the Iranian people worse off All respondents | Think the conflict will make the Iranian people worse off | Total | 36 | 55 | 19 |
| Think the conflict will make the Iranian people worse off Dem/Lean Dem | Think the conflict will make the Iranian people worse off | Dem/Lean Dem | 56 | 94 | 38 |
| Think the conflict will make the Iranian people worse off Rep/Lean Rep | Think the conflict will make the Iranian people worse off | Rep/Lean Rep | 17 | 13 | -4 |

Note: Figures for U.S. adults may differ slightly from those previously published by the Center because they are based on a subset of respondents to the original survey.

Source: Surveys of U.S. adults and “digital twins” synthetic analysis using Claude Opus 4.6 set to low reasoning with extended profile information and expert reflection. Surveys were conducted March 23-29, 2026 (replicated March 30-April 2), and April 20-26, 2026 (replicated April 27-May 1).“Can AI Stand In for Human Survey-Takers? Not Really”

Our synthetic poll consistently overstated public support for the conflict between the United States, Israel and Iran that began on Feb. 28, 2026. The model’s estimates were 10 to 19 percentage points higher than human respondents on topics ranging from overall approval of the strikes against Iran to whether the conflict will make the Iranian people better or worse off.

On this issue, too, the model was especially prone to overstating support among Republicans. In our synthetic panel, around nine-in-ten Republicans and Republican leaners approve of the strikes against Iran; are confident Trump can make good decisions about Iran policy; think the strikes were the right decision; and say the U.S. is doing enough to prevent civilian casualties. In reality, around two-thirds of Republicans hold each of these views. That is a solid majority, but hardly the unanimous agreement implied by the synthetic panel.

The model did a better job at estimating disapproval of the conflict among Democrats and Democratic leaners. But even among this group there were notable misses. For instance, 94% of Democrats in our AI poll think the conflict will make the Iranian people worse off, while in reality, that figure is 56%.

#### Data centers

### Synthetic surveys miss the mark on awareness and views of data centers

*% who have heard __ about data centers*

| Statement | Synthetic respondents | U.S. adults | Error |
| --- | --- | --- | --- |
| A little | 94 | 49 | -- |
| A lot | 3 | 25 | -- |
| Nothing at all | 3 | 26 | -- |
| Data centers are mostly bad for home energy costs | 99 | 50 | 49 |
| Data centers are mostly bad for local jobs | 0 | 20 | -20 |
| Data centers are mostly bad for local tax revenue | 0 | 17 | -17 |
| Data centers are mostly bad for people's quality of life nearby | 97 | 41 | 56 |
| Data centers are mostly bad for the environment | 98 | 53 | 45 |
| Not sure if there are data centers in their area | 97 | 44 | 53 |

Note: Figures for U.S. adults may differ slightly from those previously published by the Center because they are based on a subset of respondents to the original survey.

Source: Survey of U.S. adults and “digital twins” synthetic analysis using Claude Opus 4.6 set to low reasoning with extended profile information and expert reflection. Survey was conducted Jan. 20-26, 2026 (replicated March 9-12 and April 7-10).“Can AI Stand In for Human Survey-Takers? Not Really”

In a January 2026 survey, we asked Americans about their views on data centers – a newly salient political issue. At the time, public awareness of data centers was quite varied: 25% of U.S. adults had heard a lot about them, but a nearly identical share had heard nothing at all. Around half of Americans were in the middle, reporting that they had heard a little about data centers.

By contrast, our synthetic poll estimated that 94% of U.S. adults have heard a little about data centers, and just 3% each have heard a lot or nothing at all.

In a series of follow-up questions asked of both human and AI respondents who had heard about data centers, nearly every synthetic respondent said they weren’t sure if there were any data centers built or planned in their area – more than double the 44% of human respondents who were similarly unsure.

The AI poll also missed badly on questions about the perceived impacts of data centers. At least 97% of the synthetic respondents who received these questions said data centers are mostly bad for home energy costs, the environment and the quality of life for people who live nearby.

### Synthetic surveys overestimate some economic concerns but underestimate others

*% who are very concerned about …*

| Statement | U.S. adults | Synthetic respondents | Error |
| --- | --- | --- | --- |
| The cost of health care | 71 | 84 | 13 |
| The cost of housing | 63 | 70 | 7 |
| The price of electricity | 52 | 23 | -29 |
| The price of food and consumer goods | 66 | 78 | 12 |
| The price of gasoline | 35 | 22 | -13 |

Note: Figures for U.S. adults may differ slightly from those previously published by the Center because they are based on a subset of respondents to the original survey.

Source: Survey of U.S. adults and “digital twins” synthetic analysis using Claude Opus 4.6 set to low reasoning with extended profile information and expert reflection. Survey was conducted Jan. 20-26, 2026 (replicated March 9-12 and April 7-10).“Can AI Stand In for Human Survey-Takers? Not Really”

Although many humans hold these views, the actual shares among those who have heard about data centers range from 41% to 53%.

At the same time, the AI poll greatly *understated* the share of the public who says that data centers are mostly bad for local jobs or tax revenue.

#### Economic concerns

In a series of questions about economic challenges facing Americans today, the synthetic sample overestimated certain concerns while underestimating others.

In particular, the model estimated that just 23% of Americans are very concerned about the price of electricity. The actual share (52%) is more than double that.

---

**Next:** [How synthetic respondents express certainty and factual knowledge](https://www.pewresearch.org/data-labs/2026/09/30/how-synthetic-respondents-express-certainty-and-factual-knowledge.md)