# Synthetic Users: What They Are Good For and What They Are Not in UX Research

> Synthetic users are personas simulated by AI. They help explore hypotheses but cannot validate decisions. What NN/g found and how to use them without harm.

*Tags: ai, research, users, methods*

---


> [!info] Quick Definition
> **Synthetic users** are profiles simulated by a language model that answer questions as if they were people from a target audience. They are not real people, they have no verifiable experiences, and they do not generate behavioral data. They are useful for exploring, not for deciding.


## What are they and how are they used?

They are created with a prompt or a tool that describes the persona: age, job, goals, context. The model answers an interview, a concept, or a survey. It is fast and cheap, which makes it tempting.

The problem is what it returns. A model's answer has the shape of a user's answer, but it does not come from a real experience. It reads like an interview; in fact it is a text completion about a profile.

## What did NN/g find?

In June 2024, Maria Rosala and Kate Moran of Nielsen Norman Group compared two tools, the Synthetic Users product and ChatGPT with a detailed role prompt, against three studies the team had already run with real people. The studies were interviews with medical sales reps in Latin America, interviews with online course learners, and a concept test. Their conclusion: what synthetic users say should be treated as hypotheses, never presented as findings from real users, and UX research still needs real people.

The failures the article describes are concrete:

| Failure | What happened in the study |
|---|---|
| Sycophancy | They valued course forums that most real participants found contrived, and called an app for delivering medical samples by drone "very useful" |
| Idealized behavior | They said they had finished online courses that real learners tend to start and abandon |
| Shallow insight | They listed what makes a course engaging with no priority or detail, too shallow to act on |
| No lived experience | They have never used any product, so their "past experiences" are not reliable |

NN/g does acknowledge where they worked: when a synthetic medical rep described a typical day, the result was very similar to what real participants said. One possible reading, not from the article: a work routine is well documented online, while opinions about a new concept do not exist in any prior text.

The article also warns about organizations with low UX maturity: they are the ones most likely to see synthetic users as an ideal fix, and the most at risk of misusing them or becoming dependent on them.

## Why does it matter to a designer?

Because the temptation is strong. A team short on time may want to "validate" an idea with ten profiles in five minutes. That shortcut creates a false sense of evidence, and then the team defends a decision nobody tested.

Academic research points the same way. A 2025 study by González-Bustamante, Verelst, and Cisternas compared synthetic responses with a probabilistic public opinion survey in Chile: on trust questions the models scored very well (F1 above 0.90), but fidelity varied considerably from item to item, and the authors call for calibration before relying on these samples. Another 2025 paper (Wang, Hunt, Tang, and Joseph) starts from the observation that models show limited diversity compared with human behavior and tries fine-tuning them on a small sample of real people. Fine-tuning improves diversity and coherence, but even the best model does not reproduce the original study's statistical results, so the authors conclude this data cannot replace participants in formal analyses.

That fits the [[model-bias|biases models carry]], the [[ai-bias-in-research|biases AI brings into research]], and the [[context-engineering|care you need with the context you provide]].

## How do you use them without fooling yourself?

Rules you can follow tomorrow:

1. **Always label the source.** Never present a synthetic finding as if it came from real users.
2. **Use them before the study.** To pilot an [[ai-interview-scripts|interview guide]] or to find missing topics.
3. **Treat them as hypotheses.** Every output should become something you check with real people.
4. **Be careful with niche populations.** NN/g warns that results there can be patchy, because the model has less data.
5. **Do not use them to set priorities.** That requires behavioral data.

A useful flow: use a synthetic user to draft ten interview questions, choose four, test them with two real people, and refine. The synthetic user did not answer your problem; it helped you prepare the conversation.

The same applies to [[ai-generated-personas|AI-generated personas]]: they can be a draft, but they do not replace what you learn by talking to users.

## When should you NOT use them?

- To validate a concept or a product decision.
- For specialized audiences or groups poorly represented online, where the model has little data.
- To ask about past experiences with a product, because the profile never used it.
- When the organization has little research maturity and could treat the results as a replacement.
- When the result will go to stakeholders as if it were evidence from users.

## Example

Illustrative example: a team wants to know whether users would accept an installment payment option in a travel app. A colleague generates five synthetic users and all of them find the option "very useful." The team is about to prioritize it, but a quick review shows the profiles never mention cost or timing. They run four interviews with real travelers and find that most prefer to pay in full at booking, out of fear of a future charge.

What they learned: the synthetic answer was comfortable and false. The real interviews changed the decision.

## How do you document an experiment with synthetic users?

If you use them, leave a record another person can repeat. Five fields are enough:

1. **Model and version**, with the date of the query, because answers change when the model changes.
2. **Full prompt**, including the role and any formatting instructions.
3. **Described profile**: age, job, context. Note which attributes came from your research and which you invented.
4. **Criterion for moving to the next phase**: what would have to happen with real people for the hypothesis to stay alive.
5. **Result and decision**: which hypothesis it generated, which one you tested afterward, and what changed.

That record also protects you. If a colleague asks why the team changed direction, the answer will not be "the AI said so," but "we generated this hypothesis, tested it with four people, and these were the results."

## Common mistakes

- Presenting an AI answer as if it came from real users.
- Trusting that "all" profiles share the same opinion without asking about the model's sycophancy.
- Using them to replace interviews because of lack of time.
- Not recording which prompt and model were used.
- Believing more profiles improve the evidence: repeating a bias a hundred times does not correct it.

## Sources

- [Synthetic Users: If, When, and How to Use AI-Generated "Research" (Nielsen Norman Group, 2024)](https://www.nngroup.com/articles/synthetic-users/)
- [Emulating Public Opinion: A Proof-of-Concept of AI-Generated Synthetic Survey Responses for the Chilean Case (arXiv 2509.09871)](https://arxiv.org/abs/2509.09871)
- [Can Finetuning LLMs on Small Human Samples Increase Heterogeneity, Alignment, and Belief-Action Coherence? (arXiv 2511.21218)](https://arxiv.org/abs/2511.21218)


---

Source: https://www.fernandoux.com/en/wiki/ai/synthetic-users/