Temperature and Top-p: Creativity vs. Precision
Temperature and top-p control how much randomness an AI model's output has. What each one does, when to adjust them, and what not to expect from them.
What does temperature do?
At each step, the model scores every possible token and converts those scores into probabilities with a function called softmax. Temperature divides the scores before that conversion. A low temperature amplifies the differences between options and concentrates probability on the most likely one. A high temperature flattens them and spreads probability more evenly.
Illustrative example: three candidate tokens with scores 2, 1 and 0. Applying softmax, these are their probabilities at each temperature:
| Temperature | Token A | Token B | Token C |
|---|---|---|---|
| 0.5 | 87% | 12% | 2% |
| 1 | 67% | 24% | 9% |
| 2 | 51% | 31% | 19% |
At temperature 0.5, token A almost always wins. At 2, the other two have a real chance and answers vary more from run to run.
In practice:
- Low temperature: the output tends to repeat itself for the same input. It suits extraction, classification or code generation where consistency matters.
- High temperature: the output varies more. It suits brainstorming, name lists or alternative phrasings, but raises the risk that the model drifts.
Anthropic’s API documentation sums it up this way: a temperature closer to 0.0 for analytical or multiple-choice tasks, and closer to 1.0 for creative and generative ones.
What does top-p do?
Top-p, also called nucleus sampling, was proposed by Holtzman and colleagues in 2019 and works differently. At each step it ranks candidates from most to least likely and keeps the smallest group whose probabilities add up to at least p. It then samples from that group.
With the probabilities in the table (temperature 1) and p = 0.9, the group is tokens A and B, because 67% + 24% = 91%; token C is left out. At temperature 2 and the same p, all three would enter, because A + B add up to only 82%. If one token has 90% probability, the group can be a single token. That is why the number of candidates adapts at each step, something a fixed top-k does not do: top-k always keeps the same number of options.
The authors wanted to avoid two problems: the bland, repetitive text that comes from maximizing probability, and the unreliable text that comes from sampling the whole distribution, including its tail of unlikely options.
Which values should I use for each task?
There are no universal numbers: it depends on the model and the task. As a starting point for choosing a direction:
| Task | Direction | Why |
|---|---|---|
| Extracting data, classifying, answering in a fixed format | Low | You want the same answer every time |
| Summarizing or explaining | Medium or low | Faithful to the text, with natural wording |
| Brainstorming, names, copy variants | High | You want different options |
Always measure with your own cases. To compare two configurations rigorously, use prompt A/B testing or evaluations, not a single response.
Does temperature 0 always give the same answer?
Not necessarily. Anthropic’s API documentation says so explicitly: even with a temperature of 0.0, the results will not be fully deterministic. And even if they were, repeating the same thing does not mean being right: a model can consistently give a wrong or invented answer (see Hallucinations: Why AI Makes Things Up and How to Design for It).
For product design this matters in two places. First, do not promise in the interface that “the AI always answers the same way”. Second, if you need a stable format, low temperature helps but does not guarantee it; other techniques, such as structured outputs, do it better.
When should you NOT touch these parameters?
When the problem is quality rather than variety. If the model answers badly because it lacks context, raising or lowering temperature will not fix it. Do not adjust them to get a correct answer by chance either. Before tuning, review the prompt (see prompt-engineering), the information the model receives and the examples.
Also do not assume they exist or behave the same way on every model. According to Anthropic’s API documentation, models released after Claude Opus 4.6 do not let you set temperature: they accept 1.0 and reject any other value with a 400 error. Top-p is similar (it only accepts values of 0.99 or higher) and top-k is rejected entirely. Each provider documents its own ranges and restrictions; confirm them in the model’s documentation before you design around them.
Example
Illustrative example: a product team has two AI features. The first extracts dates and amounts from invoices to fill a form; there it wants repeatable answers, so it uses low temperature. The second proposes three alternative titles for an article; there it wants variety, so it raises the temperature. The designer does not need the math, but should know that one feature needs consistency and the other needs diversity, and that this is decided in configuration, not in the prompt text.
Common mistakes
- Changing temperature and top-p together without a clear reason.
- Believing temperature 0 guarantees a correct or identical answer.
- Using high temperature for data tasks and then being surprised the data changes.
- Testing a parameter with a single response. Compare several runs and evaluate quality.
- Thinking these parameters are “creativity” in a human sense. They are sampling rules over probabilities.
- Assuming the parameter exists on the model you use without reading its documentation.