Self-Consistency Prompting: 5 Recipes That Ship (2026)
Self-consistency prompting in 2026: five copy-paste recipes that sample multiple reasoning paths and majority-vote the answer, plus a gate for when to skip it.
On this page
Quick Answer
Self-consistency is one move: sample the same chain-of-thought prompt several times at a nonzero temperature, then return the answer the majority of runs agree on. It comes from a 2022 Google Brain paper, Self-Consistency Improves Chain of Thought Reasoning (Wang et al., 2022), and it beats a single greedy pass on any task with one correct, checkable answer. As of 2026 the rule that survives production is short: sample a few paths, vote, and only pay for it where a wrong answer costs more than the extra tokens.
Five copy-paste recipes below. Each with the receipt, why it works, and the failure mode.
Tested on 
GPT class models. Costs are order-of-magnitude, mid-2026.
Recipe 1: The canonical vote
Claim: Run the same CoT prompt N times at temperature 0.7. Tally the final answers. Return the most common one.
Solve this step by step, then end with "Answer: ".
Question: {question}
Call it N times (N=5 to start). Parse the value after "Answer:". Return the mode.
Why: A single greedy decode commits to one reasoning path and rides its mistakes to the end. Sampling several paths lets independent lines of reasoning arrive at the same answer; wrong paths scatter, right paths cluster. The majority is the signal. This is the exact method from Wang et al., and it needs chain-of-thought underneath. No reasoning trace, nothing to vote on.
Failure mode: Temperature 0 makes every sample identical, so the vote is theater. If your paths do not vary, raise the temperature until they do.
Recipe 2: Cheap self-consistency (N=3, not N=40)
Claim: The paper sampled up to 40 paths. You do not need 40. Three to five captures most of the gain.
(same CoT prompt, sampled 3 times, temp 0.7, then majority-vote the answer)
Why: Accuracy gains from more samples flatten fast. The jump from 1 to 3 is large; 3 to 5 is smaller; 5 to 40 is mostly rounding on most tasks. Cost scales linearly with N, accuracy does not. Start at 3, raise only when agreement is low.
Failure mode: On genuinely hard multi-step problems, 3 samples can split three ways with no majority. That is a signal, not a bug. See Recipe 4.
Recipe 3: Return the agreement ratio as confidence
Claim: Do not just return the winning answer. Return how many paths agreed. That ratio is a free confidence score.
Sample N=5. Answers: [7, 7, 7, 12, 7]
Return: {answer: 7, agreement: 0.8}
Why: A 5/5 sweep and a 3/2 split are not the same certainty, but a plain majority vote hides the difference. Surfacing the ratio lets downstream code trust a 0.8 and escalate a 0.4. You get calibration for free from samples you already paid for.
Failure mode: Agreement is not correctness. Five paths can agree on the same wrong answer when the model has a consistent bias. High agreement lowers the odds of a random slip; it does not fix a systematic one.
Recipe 4: Self-consistency as a guardrail
Claim: Accept the answer only when at least K of N paths agree. Below K, fall back.
if agreement >= 0.6: return answer
else: route to a larger model, or ask a human, or flag for review
Why: This turns self-consistency from an accuracy trick into a safety gate. The cases where paths disagree are exactly the cases most likely to be wrong, so disagreement becomes a cheap trigger for a more expensive fallback. You spend the big call only on the hard ten percent.
Failure mode: Set K too high and everything escalates; too low and the gate never fires. Tune K on a labeled sample, not by feel.
Recipe 5: Universal self-consistency for free-form answers
Claim: Majority vote needs answers you can compare by exact match. For summaries, code, or prose, let the model pick the most consistent draft instead.
Here are N candidate answers to the same task.
Pick the one most consistent with the others. Return only that one.
Candidates:
1) {draft 1}
...
N) {draft N}
Why: You cannot string-match two summaries that say the same thing in different words. Universal Self-Consistency (Chen et al., 2023) replaces the vote with a second model pass that reads all candidates and selects the one that agrees most with the rest. It extends the method to every task where the answer is not a single token.
Failure mode: The selector inherits the model's blind spots. If all N drafts share a hallucination, the selector will happily pick a hallucination. Universal self-consistency finds consensus, not truth.
When to skip it
Skip self-consistency on:
- Lookups and single-fact answers. One pass is enough; voting buys nothing.
- Anything latency-critical. N samples mean N times the wait unless you parallelize them.
- Creative or open-ended generation where there is no correct answer to converge on. Consistency is the wrong target there.
Use it where a wrong answer is expensive and checkable: math, classification, extraction, structured reasoning.
The one line to remember
Sample a few reasoning paths, keep the answer they agree on, and read the agreement rate as a confidence dial. Vote where being wrong costs more than the extra tokens; skip it everywhere else.
Cost to test: $0.03
Written by
Sam Q.Sam Q. writes terse, tested prompt recipes for PromptAttic. Ships them the way you would a commit message: short, verified, no preamble.
FAQ
What is self-consistency prompting?
Self-consistency is a prompting technique where you sample the same chain-of-thought prompt several times at a nonzero temperature and return the answer the majority of runs agree on. It comes from a 2022 Google Brain paper (Wang et al.) and improves accuracy on tasks that have one correct, checkable answer, because correct reasoning paths tend to cluster while wrong ones scatter.
How many samples does self-consistency need?
Far fewer than the original paper's 40. Accuracy gains flatten fast, so three to five samples capture most of the benefit on typical tasks. Cost scales linearly with the number of samples while accuracy does not, so start at three and raise the count only when the paths keep disagreeing.
How is self-consistency different from chain-of-thought?
Chain-of-thought produces one reasoning path in a single pass. Self-consistency runs that same chain-of-thought prompt multiple times and votes across the results, so it needs chain-of-thought underneath it. Chain-of-thought generates the reasoning; self-consistency aggregates several independent runs of it to filter out one-off mistakes.
When should you not use self-consistency?
Skip it on simple lookups, on latency-critical paths where you cannot afford N times the wait, and on creative or open-ended generation where there is no single correct answer to converge on. Use it where a wrong answer is expensive and checkable, such as math, classification, extraction, and structured reasoning.
Related recipes
Chain of Thought Prompting: 6 Examples That Ship (2026)
Five paste-ready chain-of-thought prompting examples for 2026, each with its failure mode, plus when reasoning out loud makes the answer worse.
LLM Temperature Settings: 6 Recipes That Ship (2026)
Six copy-paste LLM temperature and top_p recipes for 2026: the exact setting per task type, why it works, and the failure mode it prevents.
Least-to-Most Prompting: 6 Recipes That Ship (2026)
Least-to-most prompting for LLMs in 2026: six copy-paste recipes that decompose a hard problem into ordered subproblems and feed each answer forward, with the receipt, the reasoning, and the failure mode for each.