Top P: 5 Recipes That Ship (2026)

Quick Answer

Top P, also called nucleus sampling, caps the pool of candidate tokens by cumulative probability, not by count. Set top_p to 0.9 and the model samples only from the smallest set of tokens whose probabilities add up to 90 percent; the long tail of unlikely tokens gets cut before sampling. As of August 2026, both the OpenAI and Anthropic APIs ship top_p, both default it to 1.0, and both give the same advice: tune top_p or temperature, not both. Leave it at 1.0 unless you have a specific reason. Five recipes that ship are below.

Top P is one number between 0 and 1. At 1.0 the whole distribution is in play. At 0.1 only the densest 10 percent of probability mass survives. It is a filter on which tokens are eligible; temperature is a filter on how flat their odds are. That distinction is the whole game.

Recipe 1: leave top_p at 1.0, tune temperature instead

Claim: most apps never need to touch top_p at all.

Receipt (OpenAI):

from openai import OpenAI
client = OpenAI()

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize this ticket in one line."}],
    temperature=0.3,
    # top_p omitted -> defaults to 1.0. Tune ONE knob, not both.
)

Why: at top_p 1.0 the full distribution is available, and temperature already reshapes how peaked that distribution is. One control does the job. Reaching for a second control to do the same thing is where people get lost.

Failure mode: you drop temperature to 0.2 and top_p to 0.5, outputs collapse to the same three words on every call, and you spend an afternoon blaming the model.

Ship: default top_p to 1.0. Move temperature first.

Recipe 2: top_p = 0.1 to kill the tail for extraction and classification

Claim: when the task is "pick one label" or "pull this field," clamp the tail hard.

Receipt (Anthropic):

Anthropic's Messages API takes top_p the same way.

import anthropic
client = anthropic.Anthropic()

msg = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=16,
    top_p=0.1,          # sample only from the top ~10% cumulative mass
    messages=[{"role": "user",
               "content": "Classify sentiment. Reply with one word: positive, negative, or neutral.\n\nShipping was slow but the product is great."}],
)

Why: a stray hallucinated label ("mixed", "positive!") almost always lives in the improbable tail. top_p 0.1 removes those tokens from the pool before sampling ever happens, so they cannot be picked even by bad luck.

Failure mode: you leave top_p at 0.1 on a generative task and the prose turns robotic and repetitive. Low top_p is for constrained outputs, not for writing.

Ship: top_p 0.1 for closed-set outputs; pair with temperature 0 for hard determinism.

Recipe 3: top_p = 0.9 for focused-but-alive creative

Claim: for copy, names, and brainstorms you want variety without garbage.

Receipt (OpenAI):

OpenAI's Chat Completions API:

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Give me 5 product names for a budgeting app."}],
    top_p=0.9,
    # temperature left at default; tuning top_p here, not both
)

Why: 0.9 keeps the plausible-but-diverse tokens and trims only the bottom 10 percent of mass, where the truly weird tokens sit. You get range without the model wandering into nonsense.

Failure mode: pushing to 0.99 or 1.0 for creative work reintroduces the long tail and you start seeing broken words and off-topic tangents.

Ship: top_p 0.9 for creative, temperature untouched.

Recipe 4: never move top_p and temperature together

Claim: this is the one rule that fixes most "why is my output weird" tickets.

Receipt, the wrong call:

# DON'T: two randomness knobs fighting each other
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a tagline."}],
    temperature=1.3,   # crank randomness up
    top_p=0.4,         # crank the pool down
)

Why: temperature flattens the odds so unlikely tokens get a real shot, then top_p 0.4 throws most of those same tokens out. The two settings interact non-linearly and the effective behavior is hard to reason about. Both OpenAI and Anthropic explicitly recommend altering one or the other, not both.

Failure mode: unreproducible output. You tweak temperature, nothing changes the way you expect, because top_p is silently gating the result.

Ship: pick one knob per request. Set the other to its default (temperature default, or top_p 1.0).

Recipe 5: top_p 0.2 to 0.5 for steadier JSON and structured output

Claim: when a schema must parse every time, narrow the pool without going fully deterministic.

Receipt (OpenAI):

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Reply with JSON only: {\"city\": string, \"country\": string}."},
        {"role": "user", "content": "Where is the Eiffel Tower?"},
    ],
    response_format={"type": "json_object"},
    top_p=0.3,
)

Why: structured-output modes already constrain format, but a low top_p removes the rare tokens that cause a trailing comma, a smart quote, or an extra key. It is a cheap second layer of insurance on parse reliability.

Failure mode: top_p 0 (or extremely low) can make long JSON loop or truncate oddly on some models. Keep a floor around 0.2.

Ship: top_p 0.3 plus a schema-constrained output mode for JSON you have to parse.

When NOT to touch top_p

• You already tuned temperature and the output is fine. Stop. Do not add a second knob.
• You want reproducibility. Use temperature 0 and a seed where the API supports one; top_p is not the determinism control.
• You are chasing "better reasoning." Top P does not add reasoning. For step-by-step accuracy, structure the prompt instead; see the chain of thought recipes.

Top P vs temperature, in one line

Temperature changes how flat the probabilities are. Top P changes how many tokens are eligible. Same goal, different mechanism, which is exactly why you should not use both at once. The full breakdown lives in the LLM temperature settings recipes. For editing individual token odds directly, that is logit_bias, a different tool.

Docs, year-tagged: OpenAI's API parameter reference and Anthropic's Messages API reference (both 2026) document top_p, and both carry the "alter this or temperature, not both" recommendation verbatim. The Prompt Engineering Guide's settings page covers nucleus sampling in plain terms.

Cost to test: $0. Top P is a request flag, not extra tokens. You pay only for the tokens you were already going to send.