How Many Users Do You Need? Sample Size in UX Research

Why is there no single number?

Because the right number depends on what you want to know. If you are looking for problems to fix, a few participants are enough. If you want to measure what share of people complete a task, you need many more. And if you run interviews to understand motivations, the criterion is different: it is not statistical but saturation.

NN/g frames it this way: qualitative tests find problems to fix, while quantitative studies estimate population rates and times, which requires large samples (NN/g: 5 test users for qualitative and quantitative studies).

How many users for a usability test?

The rule of five is the best known. The Nielsen and Landauer model predicts that five users find close to 85% of the problems, if the probability that a user runs into a given problem is around 31%. Jeff Sauro of MeasuringU explains the same logic with an example: after testing nine users you would have seen most of the problems that affect 20% or more of people, and raising the confidence to 95% increases the required sample to thirteen (MeasuringU: Five users).

NN/g’s formula makes the trap visible: if a problem affects only 10% of people, you need about 18 users to see it with the same probability. That is why, in mature products, five users per round are not enough to find everything.

Sauro adds an important limit: the rule applies only to discovering problems. It does not work for comparing two interfaces or for estimating task time or completion rate. His approach is iterative: never more than five users at a time, but in total you might test 15 or 20 across several rounds. NN/g suggests the same: start with five, fix what turns up, and retest with a new group of about five.

How many interviews for a qualitative study?

There is no formula here. NN/g says five interviews are “often not enough,” and suggests starting with five or six, analyzing as you go, and continuing to add participants until the themes stop changing. That point is called saturation: when new interviews no longer add ideas that would change the themes (NN/g: Interview sample size).

The same article cites studies that suggest very different ranges. Guest, Bunce, and Johnson (2006) found that codes saturated around 12 interviews, and that six were enough for high-level themes. Mason (2010), analyzing the abstracts of more than 2,000 PhD theses that used qualitative interviews, found a median of 31 participants, with a range from 1 to 95. NN/g also notes that there is no academic consensus on saturation. So it helps to state a range at the start of the study for budgeting, and then let the data decide when to stop.

Factors that shift the number include the scope of the question, the diversity of the population, the interviewer’s experience, participants’ domain expertise, and how structured the conversation is. According to NN/g, a broad study of a diverse population may need 20 to 30 interviews or more, and a narrow study of a homogeneous group as few as five.

And for surveys?

For surveys and quantitative studies, NN/g usually recommends more than 30 participants, and notes that confidence intervals tend to become narrow with 40 or more. These numbers help you measure percentages with some confidence. With small samples, any figure can look important and still be noise.

When should you NOT worry about the number?

• If the decision is cheap and reversible, a test with five people and a small change may be enough.
• If the question is “is there a problem?” rather than “how many people have it?”, five people who hit it already show that it exists.
• If the team asks for an exact figure for a decision nobody will review, the sample size debate is a distraction. Define first which decision depends on the study.

Illustrative example

Illustrative example: a team tests a customer sign-up form and finds four problems across five participants. They fix all four and run a second round with five new people. A fifth problem appears, affecting only one person, but it blocks sign-up. The team does not conclude the form is fine just because the earlier problems did not come back; they run a third round focused on that case. If the goal were to measure how many people complete sign-up without help, five participants would not be enough, and they would need a quantitative study.

Before you set the number, define the research plan with its main question. And review research bias: a large sample with a recruitment bias is still a biased sample. For satisfaction measurement, the article on NPS, CSAT, and CES explains which question each metric answers.

Common mistakes

• Using five users to estimate success rates.
• Running one large test instead of several small rounds.
• Treating the rule of five as a law for every method.
• Always recruiting similar people, which makes even a large sample unrepresentative.
• Stopping interviews at the first recurring theme without checking whether others appear.
• Presenting a percentage from a sample of ten people as if it were market data.

What you can do tomorrow

1. Write down the method for your next study and ask whether you are looking for problems, motivations, or rates. That answer decides the size.
2. If you run usability tests, plan three rounds of five people instead of one round of fifteen.
3. For interviews, set a range in advance, for example six to twelve, and note when you stopped seeing new themes.

Sources

• NN/g: 5 Test Users for Qualitative and Quantitative Studies
• NN/g: Interview Sample Size
• MeasuringU (Jeff Sauro): Why you only need to test with five users