Sunday, November 1

Synthetic Research: Can AI Really Stand In for Survey Respondents?

Synthetic research: group of people seated in a market research focus group
Photo: ComputerUserUser / Wikimedia Commons (CC0)

Market research is slow and expensive. You write a survey, recruit people, wait for answers, clean the data, and pay for the whole thing. So it is no surprise that startups are exploring a shortcut: use AI to simulate the people.

Synthetic research: group of people seated in a market research focus group

This approach is often called synthetic research. Instead of asking a thousand real respondents what they think about a new snack, a price change, or an election issue, you ask a set of AI-generated personas and read their answers. It is fast and cheap, and it raises a fair question about whether the answers mean anything.

The honest answer is that it depends what you are using synthetic research for.

How synthetic research simulates people

The core is a language model, the same kind of system that powers chatbots. It has absorbed a vast amount of text written by people, so it can imitate how different kinds of people might talk. A researcher gives it a profile, say a parent in their forties living in a mid-sized city who shops on a budget, and asks it to answer a question in character.

Run that across hundreds of profiles and you get something that looks like survey data, complete with percentages and quotes. Some tools also feed in real survey results to calibrate the personas, so the simulated crowd resembles the actual population more closely.

The result is persuasive, and that is part of the problem. A tidy chart can look scientific even when the foundation is a model guessing what people typically say.

Consider a question about how much someone would pay for a streaming subscription. A real respondent thinks about their own bills, their partner, and the show they just finished. A simulated one has no bills. It only knows how such answers are usually phrased.

Where it can genuinely help

Early-stage thinking is the strongest use of synthetic research. Before spending money on a real survey, a team can use simulated respondents to test whether questions are clear, find confusing wording, and brainstorm objections they had not considered. It is a way to sharpen the survey, not to replace it.

It can also help when real participants are hard to reach or ethically sensitive. Speed is another advantage of synthetic research. A product team that needs a rough reaction to five slogan options by the afternoon can get something directional, as long as everyone remembers that it is directional.

Think of it like a rehearsal. Actors run lines before opening night, but the audience still decides whether the show works.

Some teams also use it to prepare interview guides, since a simulated participant can reveal where a question is leading or too long.

synthetic research: close-up detail of the photo, left side

Synthetic research: where it breaks down

The main weakness is that a model reflects the text it was trained on, not the actual people you want to study. It may repeat stereotypes, overrepresent the views of people who write a lot online, and miss communities that leave little digital trace. It also tends to be agreeable and consistent, whereas real people are messy, contradictory, and surprising.

New situations are another trap for synthetic research. A model can describe how people have reacted to similar things in the past, but it cannot know how they will react to something genuinely new. A product with no precedent has no training data behind it.

There is also the matter of false confidence. If a simulation says sixty percent of customers would buy a product, there is no real sample behind that number. Treating it as a measurement is where companies can get burned.

Researchers have also noted that the same prompt can give different results on different days, which makes it hard to reproduce findings, a basic requirement of reliable research.

Privacy is a smaller but real advantage in one direction: no real person has to hand over personal details. In the other direction, the training text itself came from real people who never agreed to be imitated, and that question is still debated.

synthetic research: second close-up detail, lower right

A sensible way to use it

If you work in a company considering these tools, ask a few questions. How were the personas built? Has the vendor compared simulated answers with real survey results in your area, and can they show how close they were? What happens when the model is wrong?

Use synthetic research for drafting, brainstorming, and stress-testing, then confirm important decisions with real people. Label the results clearly in reports so nobody mistakes a simulation for a sample.

A short note such as this section reflects simulated respondents is enough to keep a reader from misreading a chart.

Good practice also means keeping the prompts and settings you used, so a colleague can repeat the exercise and see whether the same patterns appear. Keep the budget for genuine interviews and surveys when stakes are high, such as pricing, health, or anything affecting customers’ money.

Synthetic research is cheap, but a wrong decision based on it is not.

For wider context on synthetic research, the Wikipedia article on survey methodology is a neutral place to start. To keep reading here, try Can an AI Interviewer Replace the Customer Call? What Teams Gain and Lose or Dual-Screen Phones: What Apps Can Do With Two Displays.

Synthetic research will likely improve, and it may become a useful part of the research toolkit. But it is a supplement. Understanding people still requires asking them.

Join the conversation

Your email address will not be published. Required fields are marked *