Design

Your capstone wraps a complete LLMOps stack around a real AI feature — ideally one you built in a prior course, so the operational work is concrete. Not a new AI capability: the operational scaffolding that makes an existing one trustworthy in production. This is the deliverable that proves you can operate AI, which is rarer and more valuable than being able to build it.

Choosing the feature

• Best: a prior-course project — triage assistant, HarborDocs, or the resolution agent. You know its behavior, so you can write a real golden set and spot real regressions.
• Fine: any AI feature with a clear input→output you can call programmatically and evaluate. If you can't specify 'correct,' you can't operate it.
• Scope it to one feature, done thoroughly. The impressive capstone isn't breadth — it's one feature with a genuinely trustworthy loop around it: evals that catch regressions, tracing that debugs, a gate that holds.

The design document (deliverable 1)

1. 1The feature + its risks — what it does, what 'wrong' costs, which failure modes matter most (this ranks everything after).
2. 2Eval plan — golden-set composition and size, the assertion and judge checks, the judge's calibration target, the holdout policy, and the numeric quality bar.
3. 3Observability plan — what each trace captures, the metrics on the dashboard, the sampled online-eval rate, the alerts.
4. 4Versioning plan — how versions are stored, the production pointer, last-known-good, rollback procedure.
5. 5Cost plan — target cost per request, the levers you'll use (caching, routing, prompt-thrift), and the budget alert.
6. 6CI & release plan — the eval gate (which checks hard-fail), the canary stages and their promotion metrics, the auto-rollback trigger.