AI Interview Scripts: Writing Questions Without Leading Users
AI drafts an interview script in minutes but tends to slip in leading questions. How to prompt for it, review it with NN/g criteria, and pilot it before use.
What does a model do well, and where should you not trust it?
A model quickly produces questions by topic, several versions of each, and follow-up examples. According to NN/g, writing a good guide is tricky and time-consuming: it is not uncommon to spend a full day on one. A first draft helps.
The evidence on quality is limited and comes from another field. A paper accepted at the 2025 IEEE requirements engineering conference (Shen, Singhal and Breaux) tested GPT-4o generating follow-up questions from what the interviewee had just said. With minimal guidance, the model’s questions were no worse than human interviewers’ on clarity, relevancy and informativeness. When the model was given common interviewer mistakes to avoid, its questions were rated higher than the human ones.
The authors themselves note limits: ratings came from people recruited on Prolific, not trained in discourse analysis; the human interviewers were graduate students with software experience, not expert researchers, which may overstate the model’s relative performance; and the study covered interviews about directory apps, with a single model. It also evaluated follow-up questions, not complete scripts. It is a sign that AI improves when it receives criteria, not proof for your project.
What mistakes does it make most often?
The most common one is the question that suggests the answer. NN/g explains that closed questions can reveal what you are interested in and prime participants to offer only certain information, and that leading closed questions often start with “did,” “was” or “is.” These pairs come from NN/g’s own guidance:
| Leading question | Open version |
|---|---|
| Was that experience helpful? | How did you find that experience? |
| Did you find that task difficult? | How did you find that task? |
| Did that make sense? | What did you think about that? |
| Was it stressful? | How did you feel about that? |
A second kind of mistake is asking about the future or about opinions. Jakob Nielsen warns that “would you use feature X?” yields no useful data because people cannot predict their behavior toward something they have not seen. NN/g recommends open questions about a specific event instead, such as “tell me about the last time you cooked at home” or “tell me about a time when you cooked something that didn’t turn out as you hoped.” Beware of “usually” too: according to Nielsen, if you ask people how they usually perform a task, they will often describe an idealized workflow; asking for recent, specific instances is more reliable.
You do not have to remove every closed question: NN/g accepts them to clarify details, sparingly and after the open ones (the funnel technique).
How do you prompt for the script and review it?
NN/g proposes seven steps for writing a semistructured interview guide. AI can help with several, but not the first:
| NN/g step | Possible role for AI |
|---|---|
| 1. Write the research questions first | None: it is the decision the script must support |
| 2. Brainstorm questions, without judging | Generate a wide pool |
| 3. Turn closed questions into open ones | Reword; you verify |
| 4. Fill gaps: each research question needs at least one interview question | Flag gaps; you decide |
| 5. Arrange: easy warm-up, reflective questions later | Propose an order; you adjust |
| 6. Prepare follow-ups and probes | Propose variants |
| 7. Pilot | None: you do it with a person |
A useful prompt includes the research goal, who the participant is, the duration, the question types you want (open, specific event, follow-up) and those you do not (closed, leading, about the future, or naming features too early). CleverX, a research-platform vendor that publishes a guide on the topic, also suggests asking for two to three times more questions than you will use, so you can choose, and asking the AI to flag any that look leading, presumptive or double-barreled.
Then review each question yourself with three filters: does it invite a story, does it suggest the answer, and can it be answered without knowing the product? The same CleverX guide warns that questions that pass AI review but fail human review are common: AI catches obvious bias and can miss subtle presuppositions in the framing.
How do you pilot it before using it?
Run a pilot with one person from your audience. According to NN/g, a pilot shows which questions are missing, which need rewording, whether the order works and whether the time is enough. Leave time to make changes afterward. The final script is not the AI’s: it is the one that survived the pilot.
This article covers only the script. If you want an AI to run the session, see AI-moderated usability tests; if you already have transcripts and need to analyze them, see AI interview synthesis. And synthetic users do not replace the pilot or the interview.
When should you not use it?
- When the script will be used as is, without review or a pilot.
- For emotional or sensitive topics, where wording matters a great deal and is better written by hand.
- When the team does not know which decision it wants to support: AI cannot define that goal (it is step 1 in NN/g’s list).
- For contexts the model knows little about: it may assume things that do not apply. This is a reasonable precaution, not a result measured in this article’s sources.
Example
Illustrative example: a designer wants to understand why people cancel scheduled trips in a ride app. Her first prompt asks for “questions to interview users about cancellations.” The draft includes “does it bother you that the app charges for canceling?”, a closed, leading question of the kind NN/g advises rewording. She rewrites the prompt with the goal, the context, the duration and the question types she does not want, and asks for triple the questions. From the new draft she picks “tell me about the last time you canceled a scheduled trip.” In the pilot with one person she finds that a timing question was confusing and rewords it before the interviews.
A 30-minute script for that case could look like this (the timings are a suggestion for this example, not a rule):
- Warm-up (3 min): “Tell me a little about yourself and how you use the app.”
- Context (5 min): “Tell me about the last time you scheduled a trip.”
- Specific event (12 min): “Tell me about a time you had to cancel. What happened before? And after?” Follow-ups: “Tell me more about that,” “Why was that important to you?”
- Decision (5 min): “When you decided to cancel, what options did you see?”
- Closing (5 min): “Is there anything I did not ask that you think is important?”
Almost everything is open, reflective questions come after the warm-up, and the product appears once the person has told their own story.
Common mistakes
- Using the first AI draft without reviewing the questions.
- Asking for a script without stating the decision it supports.
- Accepting questions about the future (“would you use…?”) or about opinions instead of specific memories.
- Mentioning product features in the first questions.
- Skipping the pilot because the script “already looks good.”
- Trusting AI to catch all of its own leading questions.
Sources
- Nielsen Norman Group, Writing an Effective Guide for a UX Interview (Maria Rosala)
- Nielsen Norman Group, Open-Ended vs. Closed Questions in User Research (Maria Rosala, 2024)
- Nielsen Norman Group, Interviewing Users (Jakob Nielsen, 2010)
- Shen, Singhal and Breaux, Requirements Elicitation Follow-Up Question Generation (arXiv 2507.02858, 2025)
- CleverX, AI for generating user research questions