Teacher & student economics
Distillation as the business case for everything you've built: the cost math with all the lines, quality tolerances by task shape, and the routing pattern that keeps the teacher in the loop.
You've been running a distillation project since Module 2 — teacher-generated data, student training, comparative eval. This module names the pattern, does its math honestly, and prepares the ship decision. First, why distillation is the adaptation pattern with board-level legibility:
- The cost math, with all the lines (the finance course would insist): current state = frontier calls × volume; future state = student serving (hosted small-model API or your GPU) + amortized training (teacher data generation + runs + your time — real numbers from your own logs, a pleasant novelty) + the maintenance line (re-distillation when the task drifts or the base improves, monitoring, the eval upkeep) + the teacher's retained role (below — it never goes to zero). For Harbor Lane's 50k/month triage: typical shape is 90-95% serving-cost reduction with single-digit-month payback. Do your own with your own logs in the lab.
- Quality tolerance is task-shaped, and triage is the friendly shape: classification with an escape hatch tolerates a 2-3 point gap gracefully — the escape route catches the student's uncertainty, and downstream review catches the rest. Generation tasks (drafting customer replies) wear their quality gaps in public; distill those with much tighter tolerances and Module 6-grade eval. Know which shape you're shipping.
- Latency is the quiet second win: small models answer in a fraction of the time, which matters exactly where distillation targets live — high-volume, in-the-loop steps where 400ms vs. 3s changes what UX is possible. Price it alongside cost; sometimes it's the real reason.
Per 1,000 triage calls, with an illustrative frontier price of $6.00 / 1,000 and a student that serves at a twentieth of that ($0.30 / 1,000, the course's twentieth-cost figure): teacher-only = $6.00. Distilled student with a 5% escape to the teacher (routing the uncertain slice) = 950 student calls ($0.285) + 50 teacher calls ($0.30) = $0.59 / 1,000 — about a 90% reduction, landing in the 90–95% band. At Harbor Lane's 50k/month that's ~$300/month teacher-only vs. ~$30/month for the cascade, ~$270/month saved; a one-time training spend of a few tens of dollars pays back in well under a month. Every number here is illustrative — the point is the shape; run it on your own logs in the lab.
The teacher never leaves: the routing pattern
Mature distilled deployments run a cascade: the student handles everything it's confident about; the escape/low-confidence slice routes to the teacher (or a human, per your task's stakes); teacher decisions on routed cases get logged as future training data — the flywheel that makes the student better each re-distillation cycle. Two disciplines keep the cascade honest: the routing threshold is tuned on eval data like any control (asymmetric costs and all — your operational courses built this reflex), and the routed-traffic rate is monitored — rising escapes are your earliest drift signal, arriving weeks before eval scores move. The pattern should feel familiar: it's the unsure lane from the automation course, wearing model weights.
Module 1's ToS check was about training; shipping adds questions: some providers' terms constrain using outputs to build competing services, and definitions vary. An internal triage model distilled from a frontier teacher is squarely mainstream practice under major providers' current terms — but 'current' and 'major' are doing work in that sentence. Confirm for your provider, in writing, at ship time. Thirty minutes with the ToS beats any conversation that starts with legal discovering the pipeline post-launch.