Self-consistency

Model outputs have randomness — the same prompt can land differently on borderline cases. Self-consistency turns that from a bug into an instrument: run the same input several times, compare the answers, and let agreement decide.

The mechanics

1. 1Run the identical prompt N times (3 and 5 are the standard budgets) — with sampling variation on, i.e. normal temperature, not zero. (temperature = how much the model varies its wording run to run; in a chat window variation is already on, so you just start each run in a fresh chat.)
2. 2Extract the decision field from each run — category, the number, the verdict.
3. 3Unanimous → accept, with earned confidence. Majority → accept the majority, mark confidence MEDIUM. Split → don't average; route to a human (Module 6 builds this path).

The insight worth internalizing: disagreement between runs is a confidence measurement you get almost for free. A single run gives you an answer; three runs give you an answer and an error bar. For classification, the 3-run agreement rate on a case predicts its real-world error rate remarkably well.

The economics

• 3× the calls = 3× the cost and latency. Never blanket-apply it.
• Worth it: decisions that trigger real actions (auto-refund? escalate to legal?), high-stakes extraction (contract amounts), and grading your own eval sets (Module 5).
• Not worth it: drafts a human reviews anyway, low-stakes routing, anything where being wrong costs less than 3× compute.
• The hybrid pattern: single run when confidence is HIGH; auto-trigger 3-run consistency only when the first run says MEDIUM/LOW or the validator flags something. Most of the benefit, a fraction of the spend.