Golden datasets: your test set of cases for AI

What is a golden dataset?

If you have read what evals are, this is the other half: the eval is the test and the golden dataset is the material it is run on. Each case has an input (what the user wrote, the state of the system, the documents available) and an expected result or an acceptance criterion. When you change the prompt, you run the set and compare.

“Golden” does not mean the answers are perfect. It means they are the reference you measure against. That is why the set is treated as a controlled artifact: if it changes silently between two runs, the comparison is worthless.

Where do the cases come from?

There are three sources, in order of importance:

1. Real production traces. Conversations that failed, answers a user corrected, cases a reviewer flagged. Langfuse treats failures as the highest-value items and recommends moving flagged traces into the set while the failure is recent. These are candidates: a person must write the correct output before they count in an experiment. It also says any class of input that has caused an incident or a support ticket belongs in the set.
2. Imports. A spreadsheet of cases the support team already keeps can be loaded as a CSV.
3. Synthetic data. Useful to cover gaps that production has not yet exercised. Langfuse’s rule is to review them with the same rigor as real items and to use them to extend a production-based core, not replace it.

Hamel Husain and Shreya Shankar point the same way. They say error analysis on real traces is the most important activity in evals, that generic metrics (helpfulness, coherence) measure abstract qualities, and that synthetic data gives only limited evidence about how real users will behave. They recommend reviewing at least 100 traces before deciding what to measure.

How do you organize it?

A common mistake is putting everything in one file. Langfuse recommends one dataset per component or behavior, without mixing the end-to-end flow with a single step, so that when a case fails the cause is localized. It suggests folder-style names such as support-agent/golden and support-agent/adversarial.

Size depends on the goal. These are Langfuse’s reference points:

To get started, Langfuse suggests 20 to 50 reviewed cases. And it stresses something more important than the number: covering distinct failure modes. Ten cases that failed for different reasons are worth more than a hundred of the same kind.

How do you keep it from going stale?

Langfuse says freshness is the property that decays silently: a set can be valid and still be old, because users, policies, tools and the language people write in all change. A reasonable process:

1. Add, do not replace. Treat the set as a log that grows and tracks production: each new failure enters as a case.
2. Date every case in its metadata, so you can see at a glance how old the set is.
3. Review old cases. Langfuse suggests a quarterly pass over cases older than six months, comparing each with the product’s current behavior. If a policy changed, the reference is wrong.
4. Archive, do not delete. Archived cases leave future runs but stay in the version history, so older comparisons remain interpretable.
5. Do not retire a case just because the system passes it. Retire it when the behavior it measured no longer exists (for example, a removed feature) or when the expected answer became wrong after a policy change. Cases that pass confirm that something still works.

How do you version it?

In Langfuse, every addition, edit, deletion or archive of cases creates a new timestamped version, and each experiment records which version it ran against. If you compare runs separated by days or weeks, pin a version: otherwise the experiment uses the latest one and you cannot tell whether the model changed or the cases did. As a baseline, Langfuse says to typically pick the version currently in production and compare each candidate against it.

If you do not use a tool that does this for you, copy the idea: a file in git with the version in its name or tag. Langfuse recommends storing at least the date added and the source of each case (for example, production-failure). Adding who labeled it is the team’s own good practice, and it helps when someone disputes a case.

Example

Illustrative example: a restaurant’s booking assistant has a golden dataset of 300 cases, split into four files: bookings, cancellations, menu questions and edge cases. When a user reports that the assistant accepted a booking for a day the restaurant is closed, the case enters the bookings file with the source “support trace” and the correct answer. When the prompt changes, the quick gate runs the 60 booking cases and finds that the change fixed that case but broke two cancellation cases. That is the value of having the set split and versioned.

Every three months, the team samples 50 recent traces and discovers that 30% of conversations are now in English, when the set had only 5%. They add English cases and note the change in the version log.

Steps to start

1. Pick one behavior of your product and collect 20 to 50 real traces of it, starting with the ones that failed.
2. Write, for each one, the reference answer or the acceptance criterion. It does not have to be perfect: Husain and Shankar advise treating initial expectations as a starting point you will revise.
3. Save it in your own file, with version and date, outside any tool. That way you do not depend on a platform: OpenAI, for example, announced the shutdown of its Evals platform.
4. Add each new failure to the file as soon as it appears.
5. Set a quarterly date to review it against production. If you use a model as a judge to grade, validate that judge against cases labeled by people (more in LLM-as-judge).

Common mistakes

• Building the set only from examples the team invented.
• Putting every case in one file, without separating by component or behavior.
• Changing the set without versioning and comparing results from different versions.
• Deleting old cases instead of archiving them.
• Believing a set of 500 similar cases is worth more than 50 that cover distinct failures.
• Using synthetic cases without reviewing them as rigorously as real ones.

Sources

• Golden dataset evaluation: build and maintain LLM test sets (Langfuse): sizes, case sources, synthetic data, versioning, quarterly review, archive instead of delete.
• LLM Evals: Everything You Need to Know (Hamel Husain and Shreya Shankar): error analysis, limits of generic metrics and synthetic data, eval sets that go stale.
• Evals guide (OpenAI documentation): test data with reference labels and the notice of the Evals platform shutdown.