Focus Groups in UX: Why They Are Rarely the Answer

What focus groups are, why they are a poor way to evaluate usability, which group biases distort their answers, and when they are still worth running.

info Quick Definition
A focus group brings a small group of users together to discuss a topic under a moderator’s guidance. It is a good method for exploring attitudes and expectations, but a poor way to find out whether a design works, because what people say in a group is not always what they do when they use the product.

What is a focus group, and what does it measure?

A focus group brings together 5 to 10 people who discuss a topic following a script. The moderator asks questions, prompts discussion, and watches how participants react to each other. IxDF describes it as a relatively expensive method whose results should be checked with more research before being taken as true.

What it measures is opinion in a social context. That is its limit: answers depend not only on what each person thinks, but on what others say in the room.

Why does this matter for a junior designer?

Because the focus group is the method most often requested out of habit. Someone says “let’s run a focus group” to validate a screen, and the result arrives as friendly comments nobody can turn into a decision. A junior who knows which question each method answers can propose a better one and explain why.

Focus groups are also expensive in time and logistics, and that cost is often paid to get an answer that a usability test would have given more precisely, with fewer people.

Why is it a poor way to evaluate usability?

Nielsen Norman Group is direct: focus groups are a fairly poor method for evaluating an interface’s usability, and they should not be your only source of usability data. Its reasons are concrete:

  • What people say is not what they do. NN/g notes that there are often large differences between the two.
  • They watch a demo, not real use. Participants see a moderator operate the product and never face the question of what to do next, which is the one that comes up in real use.
  • They opine on the concept, not the task. A group may say a feature seems useful and then fail to find it when they need it.

To find out whether someone can operate a system, NN/g considers that the only appropriate method is to observe one user at a time. Direct observation should supplement the focus group, not the other way around. If you want to review that difference, usability testing is the method that deals with the task.

What biases distort a focus group?

A group changes how people form opinions, and that can skew results in several ways:

  • Dominance. The loudest voice sets the tone, and other participants adjust their answers.
  • Conformity. Participants may align their answers with the group. Choi and Clark (AMCIS 2013) propose studying this effect in software-development focus groups; their paper is work in progress and reports no results, so treat the idea as a hypothesis.
  • Unrepresentative samples. NN/g warns that online groups attract people with above-average interest in computers, which skews the sample.
  • Unheard voices. IxDF notes that large groups are hard to facilitate and some participants end up unheard.

So when a focus group shows consensus, ask whether it is the group’s opinion or simply the dominant voice.

When is a focus group worth running?

  • To explore expectations or vocabulary. If you want to know how your audience talks about a problem, a group can give you clues you then verify.
  • To discover what matters in conversation. What worries people, what they compare, and what they find acceptable can come up naturally.
  • When the topic is about the market and usability is secondary. NN/g notes that focus groups belong more to product market research.

For these questions, user interviews are usually deeper and less biased, because there is no group pressure.

When should you NOT run one?

  • To assess whether an interface works. Usability tests are for that.
  • For behavioral questions. Asking “how do you do this?” in a group produces idealized accounts.
  • For studies that run over days or weeks. If you need to see a practice over time, a diary study or on-site observation is a better choice.
  • On a tight budget. IxDF describes it as relatively expensive because of the number of participants. If the budget is short, ask whether a few individual interviews answer your question.

Example

Illustrative example: a team wants to know whether users would accept a peer-to-peer payments app.

  • Focus group: six people agree the idea is good. When the moderator asks whether they would use it, two say yes and four nod without committing.
  • Interviews: the team runs ten interviews asking about the last time they paid a friend. It finds that most use cash or a transfer they already have, and nobody mentions a fee problem.
  • Usability test: with a prototype, participants try to split a bill. Four out of five cannot find the option to add a friend.

The idea seemed good in the group. Watching the task showed the problem the group could not see.

If your interview synthesis relies on AI, the article on AI interview synthesis explains what can be automated and what should be reviewed by hand.

Common mistakes

  • Using the group as evidence of use or usability.
  • Taking the dominant voice or an apparent consensus as the group’s opinion.
  • Relying on a single group.
  • Showing a demo instead of letting participants attempt the task.
  • Using online or power-user samples as if they represented the average user.
  • Asking about past behavior instead of observing it.

Sources

Frequently asked questions

What is a focus group in UX?
It is a session with a small group of users, usually 5 to 10 people, who give opinions under the guidance of a moderator following a script. It is useful for exploring opinions and attitudes, not for measuring how a product is used.
Why are focus groups a poor way to evaluate usability?
Because they show what people say they do, and according to Nielsen Norman Group there are often large differences between what people say and what they do. Participants also watch a demo and never face the question of what to do next in real use.
What biases affect a focus group?
A dominant voice can set the tone, and participants may adapt their answers to the group. Choi and Clark (AMCIS 2013) propose studying conformity in software-development focus groups; it is a work in progress with no results.
How many focus groups do you need?
Nielsen Norman Group recommends running more than one group, with at least six users in the first. IxDF agrees that repeating the session checks whether the questions give consistent answers.
What should you use instead to see whether a design works?
Observe one user at a time while they attempt a real task. Nielsen Norman Group considers this the only appropriate method for finding out whether someone can operate a system.