LLM Temperature Settings: 6 Recipes That Ship (2026)

Quick Answer

Temperature is the one sampling knob most people set by vibes and then blame the model when it drifts. As of 2026 the rule that survives production is boring: turn it down for tasks with one right answer, turn it up for tasks with many good answers, and change temperature or top_p, never both. Extraction, classification, code, and structured output want temperature near 0. Brainstorms, names, and copy variants want 0.7 to 1.0. Below are six copy-paste recipes, each with the setting, the reason, and the failure mode it prevents.

Temperature scales how sharply the model prefers its most likely next token. Low temperature makes the top token almost certain. High temperature flattens the distribution so unlikely tokens get a real chance. top_p (nucleus sampling) is the other lever: it keeps only the smallest set of tokens whose probabilities add up to p, then samples from that set.

Tested on Claude and GPT models. The numeric ranges differ (Anthropic caps temperature at 1.0, OpenAI goes to 2.0), but the intent maps cleanly. Reference points: Anthropic's Messages API reference and OpenAI's chat completions API reference. This card is one page from the 2026 Prompt Engineering Cheat Sheet.

The one question

Before you set a number, ask: does this task have one correct output, or many acceptable ones?

• One correct output (parse, classify, compute, format): push temperature down toward 0.
• Many acceptable outputs (ideate, rewrite, vary tone): push temperature up toward the ceiling.

Everything below is that question applied six times.

Recipe 1: Deterministic work runs at temperature 0

Claim: if there is exactly one right answer, sampling variety is not a feature, it is a bug you pay for.

temperature: 0
task: extract the invoice total and due date as JSON

Why: extraction, classification, routing, SQL, and chain-of-thought reasoning all have a target output. At temperature 0 the model takes its single most likely path, which is the one you tuned the prompt to produce.

Failure mode: leave the default (usually 1.0) on an extraction endpoint and roughly one call in twenty invents a field or reorders keys. You will spend a week writing defensive parsers for a problem a single parameter would have removed.

Recipe 2: Change temperature or top_p, never both

Claim: tuning two knobs that both shape the same distribution turns your output into noise you cannot reason about.

# pick ONE lane
temperature: 0.7   top_p: 1.0     # temperature lane
temperature: 1.0   top_p: 0.9     # top_p lane

Why: both parameters restrict the same token distribution. Move both and you cannot tell which one caused a regression. OpenAI's own API reference says to alter one or the other, not both. Default top_p to 1.0 and treat temperature as your only dial until you have a specific reason.

Failure mode: someone sets temperature 0.2 and top_p 0.5 to be safe, the output goes flat and repetitive, and three engineers argue about the prompt for an afternoon while the real cause is a double clamp.

Recipe 3: Temperature 0 is near-deterministic, not guaranteed

Claim: temperature 0 gives you greedy decoding, but a hosted API is still allowed to surprise you.

temperature: 0
# same input, run 50 times, diff the outputs

Why: temperature 0 means always take the highest-probability token, so results are stable most of the time. But batching, mixture-of-experts routing, and floating-point non-associativity mean production endpoints are rarely bit-for-bit reproducible. This matters when you build reproducible benchmarks: pin temperature to 0 and a seed if the API offers one, then still assert on shape, not on an exact string.

Failure mode: a snapshot test that pins the exact model string at temperature 0 goes green for two weeks, then flakes on a provider-side batching change and blocks a deploy that was never broken.

Recipe 4: Creative work runs hot, 0.7 to 1.0

Claim: at low temperature a model gives you the most obvious answer, and the most obvious name, tagline, or subject line is usually the worst one.

temperature: 0.9
task: give me 10 distinct product name candidates

Why: brainstorming, marketing copy, and tone variation are the tasks where you want the model to leave the safe center of the distribution. Higher temperature widens the pool of tokens it will consider, so ten candidates actually differ instead of being ten rewrites of the first.

Failure mode: run an ideation prompt at temperature 0.2 and every candidate rhymes with the first. You conclude the model is bad at creativity when you simply asked it to be predictable.

Recipe 5: Sample diverse, then vote

Claim: for a hard question with one right answer, one hot sample is a gamble but many hot samples plus a vote is a method.

temperature: 0.7
n: 5              # or loop 5 calls
# then majority-vote the final answers

Why: self-consistency needs variety between samples to work, so you deliberately raise temperature to around 0.7, generate several independent attempts, and take the answer that shows up most. The diversity that would be a bug in Recipe 1 becomes the mechanism here.

Failure mode: run the samples at temperature 0 and all five come back identical, so the vote is unanimous and meaningless. You paid five times for one answer.

Recipe 6: Use top_p to cut the tail, not to be timid

Claim: top_p is the right tool when the problem is rare garbage tokens, not overall creativity.

temperature: 0.8   top_p: 0.9
task: write varied copy but never emit broken markup

Why: top_p was introduced to fix the long, low-probability tail that produces incoherent text at high temperature. Keeping top_p slightly below 1.0 chops off that tail while temperature still controls variety inside the kept set. Reach for it only after temperature alone gives you variety with occasional junk.

Failure mode: set top_p to 0.5 hoping for safety and you amputate most of the distribution, so the output collapses to the same three phrasings no matter what temperature says.

Which setting to reach for

task                                  temperature   top_p
extract / classify / route / SQL      0             1.0
structured JSON output                0             1.0
step-by-step reasoning                0 to 0.3      1.0
summarize a document                  0.2 to 0.4    1.0
draft an email in a set tone          0.4 to 0.6    1.0
brainstorm names / taglines           0.8 to 1.0    1.0
self-consistency sampling             ~0.7          1.0
high variety, no junk tokens          0.7 to 0.9    0.9

Start every new endpoint at temperature 0, raise it only when the task genuinely has more than one good answer, and leave top_p at 1.0 until you can name the exact tail you are trying to cut.

FAQ

What is the best temperature for an LLM?
There is no single best value. Use temperature 0 for tasks with one correct output (extraction, classification, code, structured data) and 0.7 to 1.0 for tasks with many acceptable outputs (brainstorming, copy, tone variation). Default new endpoints to 0 and raise only when you can justify it.

Can I set the LLM temperature to 0?
Yes, and you should for deterministic work. Temperature 0 uses greedy decoding, always picking the highest-probability token. Just know that hosted APIs are not guaranteed bit-for-bit reproducible even at 0, because of batching and floating-point effects, so assert on output shape rather than an exact string.

Should I change temperature and top_p at the same time?
No. Both parameters constrain the same token distribution, so moving both makes regressions impossible to diagnose. Pick one lane, keep the other at its default (top_p 1.0), and treat temperature as your primary dial.

What does top_p actually do?
top_p, or nucleus sampling, keeps only the smallest set of most-likely tokens whose probabilities sum to p, then samples from that set. It is best used to trim the low-probability tail that causes incoherent output at high temperature, not as a general creativity control.

Cost to test: about $0.02. Take one extraction prompt, run it 20 times at temperature 1.0 and 20 times at 0, and diff the outputs. The difference in your parser workload is the whole argument.