Workshop: The eval plan

This workshop produces the artifact that makes your PRD enforceable: the quality contract plus the eval plan that operationalizes it.

1. 1Decompose and rubric: 3-5 dimensions for your feature, each with pass/fail criteria and two worked examples (write the examples from your prototype outputs — you have real material; use it). Test one rubric on a colleague or your AI assistant as the naive grader: where they hesitate, tighten.
2. 2Set the per-dimension bars, argued from the error map: each bar with one sentence of justification tied to user-felt cost ('correct ≥98% because a wrong proactive message is our expensive quadrant'). Mark which dimension is your hardest to measure and write its human-panel plan (who, how many cases, what cadence) instead of pretending a judge will hold it.
3. 3Seed the golden set (30-50 cases): your prototype's 20 (including every caveat case), 10 from each expensive error-map quadrant (write them — fabricating hard cases is a skill; your AI assistant generates candidates, you curate), and the distribution note ('production is ~70% single-package delays; set matches'). One caution: the assistant produces plausible hard cases, not necessarily real ones — same distribution trap Module 2 flagged for the prototype inputs. Validate the generated cases against real examples (support tickets, past incidents) before they earn a permanent seat; a golden set full of hard cases that never actually happen measures a feature nobody has. Define the growth rule: what gets added, by whom, from what sources, monthly.
4. 4Spec the judge and its receipts: which dimensions get judged automatically, the calibration ritual (sample size, agreement target, cadence), and the dashboard's four views (per-dimension trend, failure list, set composition, judge-agreement history). Then assemble the one-page quality contract — dimensions, bars, signatures line, the no-bar-moves sentence — and staple the eval plan behind it. This goes verbatim into your capstone PRD.