Lab: Production prep

Final infrastructure lab: HarborDocs gets its production posture. Nothing new conceptually — this is the discipline pass, and the artifacts feed straight into your capstone.

Step 1 — Latency pass

1. 1Add per-stage timing to the trace log. Run 20 queries; produce your own 'where the time goes' table with p50/p95 per stage.
2. 2Apply the top two levers the table justifies: typically a max 150 words schema cap plus streaming (stream=True in the SDK, print tokens as they arrive), and the skip-rewrite-when-no-history branch.
3. 3Re-measure. Record before/after p50 and time-to-first-token in LAB-NOTES.

Step 2 — Cost pass

1. 1Compute per-query cost from traces (token counts × your providers' prices, per stage). Publish the number.
2. 2Add the answer cache: dict keyed on (normalized query, index version), storing only HIGH-confidence verified answers. Replay your 20 queries with 5 repeats mixed in; report the hit rate and the blended cost.
3. 3Reorder your prompt static-first and note the cacheable-prefix fraction — even without enabling provider caching today, the structure is now ready for it.

Step 3 — The dashboard and the drill

1. 1Write dashboard.py: read all traces, print the one-screen report — score distribution, no-good-chunk rate, verification failures, confidence mix, latency p50/p95, cost, top abstained queries.
2. 2Run the weekly loop once for real: sample and classify 10 traces, move at least one finding into the golden set, and write the abstained-queries list as if handing it to a docs owner.
3. 3Tabletop one incident: 'verification failures spiked at 9am'. Walk your artifacts — which dashboard line, which trace field, which manifest diff — to the root cause. Time yourself; under ten minutes means your logging earns its keep.

Step 4 — The runbook

One page: current index version + eval scores, model pins (all three), corpus source + rebuild cadence, FLOOR/thresholds and their calibration date, budgets + alert lines, the weekly loop owner, and the three most-likely incidents with first responses. Staple it to the front of the project. This page plus your scorecard is 'production-ready' — not a feeling, a folder.