OpenAI Prompt Caching: 4 Recipes That Ship (2026)

Quick Answer

OpenAI prompt caching is on by default and there is no flag to turn it on. Whether you get a hit is structural, not financial: the rendered prefix has to be byte-identical, it has to clear a minimum-token floor, and it has to survive a list of invalidating request settings that has nothing to do with your prompt text. So the whole job is putting stable content first, volatile content last, and then verifying with usage.prompt_tokens_details.cached_tokens instead of assuming. Four recipes below, plus the settings that silently void a prefix and the rounding rule that makes your hit rate look worse than it is.

The one structural difference worth internalising

On Anthropic you place cache breakpoints yourself with cache_control, and if you place none you cache nothing. On OpenAI the default is the inverse: caching is implicit, the service places a breakpoint on the latest message for you, and your job is to avoid breaking a cache you already have.

That flips the failure mode. On Claude the common bug is a cache you never created. On OpenAI the common bug is a cache you keep destroying, usually without touching a single word of the prompt.

If you want the Anthropic side of this, the mechanics and the cost math are in our Claude prompt caching recipes. The rest of this page is the part that does not transfer.

The invalidation list that has nothing to do with your prompt

This is the section that is missing from almost every write-up on the subject. Per the OpenAI prompt caching guide (read September 1, 2026), a cached prefix stops matching when any of these change:

Two of those are the ones that actually cost people money in production.

Tool ordering. If you build your tool array by iterating a dict, a set, or a registry that is populated at import time, the order can vary between processes. Same tools, same schemas, different serialisation, zero cache hits on half your fleet. Sort the array once, at the boundary, and never sort it again downstream.

Per-request reasoning.effort. The natural design is to bump effort when a query looks hard. That is a cache fork: easy traffic warms one prefix, hard traffic warms another, and neither reaches the steady state you sized your budget on. Pick effort per route, not per request.

Also note the image rule, which is easy to miss: images are cached, but the detail parameter has to be identical across requests, per Microsoft's Azure OpenAI prompt caching page (updated August 2026). One request at detail: "auto" and the next at detail: "high" is a miss even with the same image bytes.

Recipe 1: Stable prefix, volatile suffix

The only layout rule that matters. Everything reusable goes at the front, in an order that never changes, and everything per-request goes at the back. Python below, but the ordering rule is language-agnostic.

# Python, openai SDK, September 2026
from openai import OpenAI

client = OpenAI()

# Built ONCE at module load, never reordered, never f-string'd per request.
TOOLS = sorted(load_tools(), key=lambda t: t["name"])
SYSTEM = open("instructions.md").read()   # at least 1,024 tokens

def answer(user_msg: str, doc: str):
    resp = client.responses.create(
        model="gpt-5.6",
        tools=TOOLS,
        input=[
            {"role": "system", "content": SYSTEM},   # stable
            {"role": "user", "content": doc},        # stable per document
            {"role": "user", "content": user_msg},   # volatile, LAST
        ],
    )
    return resp.output_text

The mistake this prevents: interpolating anything per-request into the system prompt. A timestamp, a request id, a user's display name, a "today is ..." line. One variable token at position 12 voids everything after it, and a cache that never hits looks exactly like a cache that is not configured.

If your system prompt genuinely needs the date, put it in the last message, not the first.

Recipe 2: Verify the hit, do not assume it

Caching is silent in both directions. There is no error when it misses, so an unverified cache is a guess.

# Python, September 2026
def answer_verbose(user_msg: str, doc: str):
    resp = client.responses.create(
        model="gpt-5.6",
        tools=TOOLS,
        input=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": doc},
            {"role": "user", "content": user_msg},
        ],
    )
    d = resp.usage.input_tokens_details
    cached = getattr(d, "cached_tokens", 0) or 0
    written = getattr(d, "cache_write_tokens", 0) or 0
    total = resp.usage.input_tokens
    print(f"cached={cached} written={written} of {total} input "
          f"({100.0 * cached / max(total, 1):.1f}% reused)")
    return resp

Log cached / input as a ratio per route and alert on it, because that ratio is the only signal that tells you a deploy broke your prefix. A schema tweak that changes nothing about your output will show up here as a hit rate that fell off a cliff on Tuesday.

On the Chat Completions shape the same numbers live under usage.prompt_tokens_details.cached_tokens.

Recipe 3: Route high-volume traffic across cache keys

prompt_cache_key is the parameter people reach for expecting a guarantee. It is not one. It influences which machine your request lands on, alongside current load and a hash of your leading tokens, and OpenAI's guide is explicit that keys "do not pin requests to a machine or guarantee a cache read hit."

There is a concrete ceiling, and it is the most actionable number on this page. Azure's documentation states that if requests sharing one prefix and one prompt_cache_key exceed roughly 15 requests per minute, some of them will miss. Above that you partition:

# Python, September 2026. Stable mapping, not random spraying.
import hashlib

SHARDS = 8   # raise until per-shard rate sits under ~15 rpm

def cache_key(tenant: str, prompt_version: str, request_id: str):
    h = hashlib.sha256(request_id.encode()).hexdigest()
    shard = int(h[:8], 16) % SHARDS
    return f"{tenant}:{prompt_version}:{shard}"

Two properties matter here and both are easy to get wrong. The mapping has to be stable, so a given request family always lands on the same shard; a random shard per request is strictly worse than no key at all. And the key should carry a prompt version, so that shipping a new system prompt moves traffic to fresh keys instead of fighting the old prefix for the same slots.

Recipe 4: Explicit breakpoints, and the arithmetic behind them

On GPT-5.6 and later you can stop relying on the implicit breakpoint and mark the end of a reusable prefix yourself. Useful when a long stable document sits in the middle of the request rather than at the very front.

{
  "model": "gpt-5.6",
  "prompt_cache_key": "tenant:acme:manual-v2",
  "prompt_cache_options": { "mode": "explicit", "ttl": "30m" },
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "input_file",
          "file_id": "file_abc123",
          "prompt_cache_breakpoint": { "mode": "explicit" }
        },
        { "type": "input_text", "text": "Summarize the failure modes." }
      ]
    }
  ]
}

The limits, from Azure's page (August 2026), because they decide how you structure a long conversation:

• At most 4 new cache writes per request.
• In implicit mode the automatic breakpoint on the latest message consumes one of those slots, leaving 3 for your explicit ones.
• Breakpoints from earlier turns are read-only. They can match, but they are not rewritten.
• Up to the latest 50 breakpoints are considered for reads.

ttl accepts exactly one value, 30m, which is also the default, and it sets a minimum lifetime rather than a storage policy. Setting it is documentation for your future self, not tuning.

One trap: models before the GPT-5.6 family do not accept prompt_cache_options or prompt_cache_breakpoint at all. They return HTTP 400. If you support a mixed fleet, gate these fields on model family rather than sending them everywhere and hoping they are ignored.

Two floors that are not what you think

The minimum differs by platform, and the vendors do not agree. OpenAI's guide states 1,024 tokens for GPT-5.6 and later, and 2,048 visible input tokens for earlier models, noting some may cache shorter prefixes. Azure's page states a flat 1,024 minimum and that the first 1,024 tokens must be identical. If you are porting a prompt between the direct API and Azure, the prefix you sized against one floor may sit under the other. Measure your own prefix; our token counting recipes cover doing that before you ship rather than after.

The 128-token rounding rule. On GPT-5.5 and earlier, cache hits past the first 1,024 tokens land in 128-token increments, so reported cached_tokens is rounded down and understates a real hit by up to 127 tokens. On GPT-5.6 and later the reported boundary is exact. This matters when you are debugging a hit rate that looks like 94% and is actually 100%: on an older model, do not chase the last sliver.

The costs, in one row, as supporting evidence

Deliberately last, because none of the decisions above should rest on it. Multipliers are relative to your standard uncached input rate, per OpenAI's guide read September 1, 2026:

On GPT-5.6 and later, writing a prefix once and reusing it once costs about 1.35x its ordinary input cost, so break-even is the second read, not the first. That is a change from the older models, where writes were free and the first read was already pure upside. Rates move; check the vendor page before you quote these in a budget.

When NOT to use it

Do not restructure a prompt for caching when your traffic is sparse. A prefix that goes cold between requests is a write tax with no reads behind it, and on GPT-5.6 and later that tax is real rather than zero. If a given prefix is not being reused within its lifetime by a steady stream of requests, the correct move is to leave the prompt in whatever order reads best and spend the effort somewhere else.

Also skip it when your prefix genuinely cannot be stabilised, for example a per-user prompt assembled from short, entirely personal context. Below the token floor there is nothing to cache, and above it a prefix that changes every request is just a cache write you pay for once per user.