logit_bias: 4 Recipes That Ship (2026)

Quick Answer

logit_bias is the one OpenAI parameter that edits token probabilities directly. You pass a JSON object that maps token IDs to a number from -100 to +100, and that number is added to the token's logit before sampling. A value near -100 makes a token almost impossible; a value near +100 makes it almost mandatory. It is the difference between asking a model not to say a word and making the word unreachable at the sampler. As of 2026 it works on OpenAI Chat Completions models, is ignored by the o-series reasoning models, and has no equivalent in Anthropic's Messages API. Four recipes below, each with the receipt, the reason, and the failure mode.

One catch eats most first attempts. The map is keyed by token IDs, not words. Get the tokenizer wrong and you bias random tokens while nothing errors. Every recipe here assumes you encoded the string with the model's own tokenizer. Reference points: OpenAI's Chat Completions API reference for the parameter, and tiktoken for the token IDs.

Recipe 1: Ban a token at the sampler, not in the prompt

Claim: "Never mention X" leaks. A -100 bias does not.

logit_bias: { 8202: -100 }   # the token id for the word you want gone

Why: a prompt instruction competes with everything else in context. A -100 bias subtracts 100 from the token's logit before sampling, so it loses to almost anything. Reach for it on banned brand names, a competitor you cannot name, a stray second language, a stray closing tag.

Failure mode: you bias one id, but the word is two tokens, so half of it still prints. Bias every piece, including the leading-space and capitalized variants. " Apple", "Apple", and "apple" are three different ids.

Ship: encode the exact string with the model tokenizer, bias all variants to -100, run the prompt 20 times, grep for the word.

Recipe 2: Pin a classifier to its label set

Claim: a yes/no or A/B/C/D task should be physically unable to answer anything else.

logit_bias: { 9642: 6, 2822: 6 }   # ids for "yes" and "no"
max_tokens: 1

Why: small positive nudges on the allowed tokens plus max_tokens 1 collapse the output space to your labels. You stop parsing "Sure, the answer is yes!" and start reading one token. This is also how you keep a model-grading eval honest: force the judge into a fixed label set so its verdicts stay comparable. If you run head-to-head model benchmarks, pinning the judge's labels this way is what makes two runs count the same.

Failure mode: people slam +100 on several tokens, the model is forced to emit only those, and a two-token label like " maybe" comes out garbled. Use +3 to +8 for nudging, and save near +100 for single-token labels.

Ship: keep each label to one token where you can; nudge, do not detonate.

Recipe 3: Wrong token IDs is why it "doesn't work"

Claim: most logit_bias bug reports are tokenizer mismatches, not API bugs.

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
enc.encode(" yes")   # -> the id you bias; note the leading space

Why: gpt-4o uses the o200k_base encoding; older models use cl100k_base. A token id copied from a 2023 blog points at a different token on today's models. The forums full of "logit_bias not working" are almost all this one mistake.

Failure mode: you reuse ids across model families and quietly bias punctuation. Nothing throws. The output just ignores you.

Ship: recompute the ids with the model's own encoding every time you switch models.

Recipe 4: Know when it is the wrong tool

Claim: logit_bias is an OpenAI Chat Completions feature, not a universal knob.

# OpenAI o-series reasoning models: logit_bias ignored
# Anthropic Messages API: no logit_bias parameter at all

Why: OpenAI's reasoning models drop most sampling controls, and Anthropic never exposed logit_bias. On those models you steer output a different way: stop sequences to cut generation, or structured outputs and prefill to shape it. Anthropic documents its own knobs in the Messages API reference.

Failure mode: you ship a logit_bias guardrail, swap the backend to Claude or an o-series model, and the guardrail silently evaporates. Test the guard on the model you actually deploy.

Ship: treat logit_bias as OpenAI-only, and keep a prompt-level fallback for everything else.

When to reach for it

goal                              lever
ban a specific word / token       logit_bias -100 on every variant
force a small label set           logit_bias small + on labels, max_tokens 1
stop at a delimiter               stop sequences (all models)
shape the whole structure         structured outputs / prefill

logit_bias edits single tokens; temperature and top_p reshape the whole distribution. For anything bigger than a token, a stop sequence or a schema is the better tool.

FAQ

What does logit_bias do in the OpenAI API?
It adds a fixed number, from -100 to +100, to the logit of specific tokens before sampling. A value near -100 makes a token almost impossible to generate; a value near +100 makes it almost certain. You pass it as a JSON object mapping token IDs to those values.

Why is my logit_bias not working?
Almost always wrong token IDs. logit_bias is keyed by token IDs from the model's tokenizer, not by words. gpt-4o uses the o200k_base encoding while older models use cl100k_base, so IDs are not interchangeable. Re-encode the exact string, including any leading space, with the current model's tokenizer.

What is the range of logit_bias values?
-100 to +100. Values near -100 effectively block a token, values near +100 effectively force it, and small single-digit values nudge probability without dominating. Most production uses live at -100 for banning and +1 to +8 for gentle steering.

Does Claude or Anthropic support logit_bias?
No. Anthropic's Messages API has no logit_bias parameter. To steer Claude's output, use stop sequences, prefilled assistant turns, or structured prompting instead. OpenAI's o-series reasoning models also ignore logit_bias.

Cost to test: about $0.01. Encode one token, bias it to -100, and run a prompt that would normally say it 20 times. If it never appears, the scalpel works.