Lab: Close the loop

Everything converges: the SFT(+DPO) student, quantized and served, measured against the teacher, wrapped in the cascade, priced from your own receipts. The deliverable is the ship packet a real team would take to review.

1. 1Produce the serving artifact: merge/export the tuned model, quantize (4-bit is the default claim to test), and re-run the full eval suite on the quantized artifact — the three-way table one last time (ceiling / student-as-shipped / teacher), by class, with schema, escapes, adversarial, and general slices. This is the table every claim below cites.
2. 2Tune the cascade threshold on data: sweep the student's escape/confidence threshold across the eval set and plot the trade — % of traffic routed to teacher vs. end-to-end accuracy of the cascade. Pick the operating point with the error-cost asymmetry argued explicitly (mis-routed urgent complaint vs. teacher-call cost — you've made this exact argument in three courses; make it with model weights now). State the expected teacher-traffic rate; it's a line in the cost model and your drift canary in monitoring.
3. 3Build the cost model from your own logs: teacher-only baseline at 50k/month; cascade cost (student serving at your chosen road's real prices + teacher slice + amortized training from your actual logged spend + a maintenance line you defend — re-distillation cadence assumption stated). Compute payback. Show the sensitivity: at what monthly volume does this stop being worth it? (There is one; finding it honestly is the analysis.)
4. 4Write the ship packet (two pages): the recommendation with tolerance met or not-met stated plainly; the three-way table; the cascade design with its threshold argument; the cost model with payback and sensitivity; the ops plan (registry entry, regression gate, escape-rate monitor, rollback, re-distillation trigger: 'escape rate sustained >X% or eval drop >Y points'); and the risks register (teacher ToS confirmation, drift exposure, the single-vendor question for your serving road).
5. 5The honest final check: re-run the held-out 30 from Module 1 — untouched since the day you built them — as the last number in the packet. If it disagrees materially with your eval-set results, you have a contamination or overfitting story to investigate before shipping, and finding it here rather than in production is the reason those 30 existed. Either way: that number, whatever it is, goes in the packet unedited.