Canary releases

The gate confirms a version is good offline. Production is the real test, and the disciplined way to face it is not all-at-once — it's the canary: release the new version to a small slice of live traffic, watch it closely, and promote it only if the real-world metrics hold. It's the online complement to the offline gate, and the operational form of the staged rollout you learned for agents.

The staged rollout

1. 1Canary (1–5%): route a small fraction to the new version. Watch the sampled-quality score, error/validator-rejection rate, cost, and latency against the current version on the same window.
2. 2Expand (25%, 50%): if the canary holds, widen the slice, still comparing. Each stage is a checkpoint, not a formality.
3. 3Full (100%): promote to production; the old version becomes last-known-good, kept warm for rollback.
4. 4Advance only on evidence. Each stage's promotion is gated on metrics, not on the calendar or a hunch. A canary that looks worse doesn't get 'a little more time' — it gets rolled back.

What to watch — and automate

• Compare canary vs. control on the same live window — same traffic mix, same time, so a metric difference is the version, not the hour of day.
• Watch the leading indicators: validator-rejection rate and sampled-judge score move before user complaints do. A canary going bad shows up here first.
• Automate the rollback trigger. If the canary's key metric breaches a threshold, auto-revert the slice and page — you don't want a bad canary depending on someone watching a dashboard at 3am. This is your kill-switch/rollback machinery (Module 4), fired by a metric instead of a human.

Why canary beyond the gate

The offline gate can't see everything: the true input distribution, real user reactions, downstream effects, load-dependent behavior. The canary catches the regressions your golden set didn't imagine — precisely the long-tail failures that define the reliability gap. Gate + canary is defense in depth for releases: the gate stops the regressions you can foresee, the canary limits the blast radius of the ones you can't. In the lab you'll simulate the canary with replayed traffic (a fraction of recorded requests routed to the new version); wiring it to a live traffic split is the same pointer logic behind a router — the version pointer from Module 4, chosen per-request by a hash instead of set globally.