Ship

A stack that never ran against real traffic isn't operated — it's rehearsed. This lesson takes your capstone from 'built' to 'live and operable': the go-live checklist, the runbook that lets someone else run it, and the honest answer to 'how do you know it's healthy?'

The go-live checklist

• Eval gate green on the shipping version, with the score recorded.
• Tracing on and confirmed writing complete traces in the live path.
• Dashboard live with the quality proxy, cost, and latency wired to real data.
• Rollback rehearsed — you've flipped the pointer and back at least once.
• Alerts armed — quality-drop, cost-breach, and error-spike alerts point somewhere a human sees.
• Canary configured — the first real release goes through it, not straight to 100%.

The operational runbook (deliverable 2)

One page, so someone who isn't you can operate it: current version + eval score; the golden set location + cadence; the dashboard + what healthy looks like (target ranges for quality, cost, latency); how to deploy a change (PR → gate → canary → promote); how to roll back; the top three likely incidents with first responses; and the escalation owner. If on-call can't deploy, revert, and diagnose from this page, the feature is running but not operated.

The weekly loop (the heartbeat)

1. 1Sample traces — random + low-quality + flagged — and read them. Classify failures.
2. 2Feed findings where they belong: bad outputs → golden set; drift → refresh the set; cost creep → the levers; a new failure class → a new check and maybe a new alert.
3. 3Re-run the full eval and read it next to production metrics. Offline-fine-but-production-degrading means your golden set has drifted from reality — refresh it. This divergence check is the single most important ongoing signal.