count_tokens: 5 Recipes That Ship (2026)

Quick Answer (2026): Count tokens before you send them. Anthropic exposes a dedicated endpoint, POST https://api.anthropic.com/v1/messages/count_tokens, that takes the exact same body as the Messages API and returns {"input_tokens": N}. It is free, it runs no inference, and it has its own rate limits that are separate from message creation (Anthropic token counting docs, 2026). Use it to pre-flight against the context window, to estimate a bill before a batch job, and to re-measure after a model change. It counts the whole request, not just the user text, so the number is only right if you send it the whole request.

The mistake almost everyone makes is guessing. They divide characters by four, or reuse an OpenAI tiktoken count, and then a long job dies with a 400 on token 200,001. There is no accurate public tokenizer for Claude , so the count endpoint is the only precise answer. Five recipes that ship.

Recipe 1: Pre-flight against the context window

Claim: Ask for the count first, refuse or trim before you ever hit the model.

Receipt:

curl https://api.anthropic.com/v1/messages/count_tokens \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "system": "You are a careful editor.",
    "messages": [{"role": "user", "content": "...your full prompt..."}]
  }'
# -> {"input_tokens": 14}

Why: A cut-off job wastes the whole request, and a retry pays for the prefix twice. One cheap call up front tells you whether the prompt fits, so you can trim, chunk, or route to a bigger-window model on purpose instead of by accident.

Failure mode: Counting only the user message and forgetting the system prompt and tools. That undercount is exactly what puts you over the ceiling in production.

Ship: Gate every large request on input_tokens under your model's context limit, minus the max_tokens you plan to reserve for the reply.

Recipe 2: Count the whole request, not just the text

Claim: Tools, system prompts, images, and PDFs all cost tokens. Count them the way you will send them.

Receipt: The same prompt with one tool definition jumps from 14 tokens to 403.

import anthropic
client = anthropic.Anthropic()

resp = client.messages.count_tokens(
    model="claude-sonnet-5",
    tools=[{
        "name": "get_weather",
        "description": "Get the current weather in a given location",
        "input_schema": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"],
        },
    }],
    messages=[{"role": "user", "content": "What is the weather in San Francisco?"}],
)
print(resp.input_tokens)  # 403

Why: Tool schemas, a long system prompt, and images (an image can be 1,000+ tokens on its own) usually dominate the count. A local len(text) / 4 heuristic sees none of that.

Failure mode: Estimating from the user turn alone, then being surprised when the real bill is triple your guess.

Ship: Build the count request from the identical object you pass to messages.create, tools and all. Anthropic confirms the endpoint accepts system prompts, tools, images, and PDFs (docs, 2026).

Recipe 3: Estimate the bill before a batch job

Claim: Count once per row, multiply by your input price, and you know the input cost before you spend a cent.

Receipt:

INPUT_PRICE_PER_MTOK = 3.00  # check current pricing for your model

total_in = 0
for row in dataset:
    n = client.messages.count_tokens(
        model="claude-sonnet-5",
        messages=[{"role": "user", "content": row["prompt"]}],
    ).input_tokens
    total_in += n

print(f"input tokens: {total_in:,}")
print(f"est input cost: ${total_in / 1_000_000 * INPUT_PRICE_PER_MTOK:,.2f}")

Why: Finance likes a number before the run, not after. Counting is free, so a dry run over 10,000 rows costs nothing and tells you if the job is 5 dollars or 500.

Failure mode: Forgetting output. This estimate is input only. Your reply length is a separate lever you set with max_tokens, and repeated prefixes are cheaper once you turn on prompt caching.

Ship: Add the output estimate (rows * max_tokens * output_price) to the input estimate for a full pre-run budget.

Recipe 4: Recount when you switch models

Claim: A model bump can move your token count 30 percent. Never reuse an old count.

Receipt: Count the same request twice and compare.

body = {"messages": [{"role": "user", "content": open("prompt.txt").read()}]}
old = client.messages.count_tokens(model="claude-3-7-sonnet-20250219", **body).input_tokens
new = client.messages.count_tokens(model="claude-sonnet-5", **body).input_tokens
print(old, new, f"{(new/old - 1)*100:.0f}% change")

Why: Claude 4.7 and later use a newer tokenizer that produces roughly 30 percent more tokens for the same text (docs, 2026). A prompt that fit and a cost you quoted on an older model can both be wrong after migration.

Failure mode: Sizing a new model's context window, or quoting a customer, on a count measured against last year's model.

Ship: Re-measure every prompt against the exact model string you will ship, and re-derive both context fit and cost from that number.

Recipe 5: Guard for free, on a separate meter

Claim: Token counting is free and its rate limit is independent, so you can pre-flight as aggressively as you like without eating your Messages budget.

Receipt: From the docs (2026): counting is "free to use," and "token counting and message creation have separate and independent rate limits." The per-minute request limits scale with your usage tier (2,000 on Start, up to 8,000 on Scale).

Why: A guard that costs money or steals throughput from real requests gets ripped out. This one costs neither, so it can live in the hot path.

Failure mode: Expecting the count to reflect a prompt-caching discount. It does not. Counting returns a plain estimate with no caching logic, even if you include cache_control blocks; the discount only applies during real message creation.

Ship: Put the count call inline before every user-facing generation. It is free, it is on its own meter, and it turns a class of 400s into a graceful trim.

When NOT to count

If your prompt is tiny and fixed, a one-time manual count is enough; skip the per-request round-trip. And remember the number is an estimate: the docs note the real input count "might differ by a small amount," so budget a little headroom rather than gating at the exact ceiling. For anything variable, long, or expensive, count every time.

Cost to test: $0. The count_tokens endpoint is free and never runs inference. If you want a quick sanity check without writing code, Simon Willison's Claude token counter calls the same API in the browser, and OpenAI users can do the offline equivalent with tiktoken.