Running a fine-tune (hosted or local)
The two roads — hosted APIs vs. local LoRA — the run protocol that keeps experiments honest, and evaluation as the only definition of done.
Mechanics day. Both roads lead to the same artifact (a model that behaves like your dataset); choose by constraints, not identity:
- Hosted fine-tuning APIs — a managed fine-tuning service: several providers offer LoRA fine-tuning as an upload-JSONL-set-knobs-poll flow. Upload the training file, set the few exposed knobs, receive a tuned model behind an endpoint. Pros: zero infrastructure, hours not days, serving included. Cons: base-model menu is the provider's, knob access is partial, and your training data transits their service (check terms; your Module 2 hygiene already made the file clean to share). Right answer for: teams without GPU muscle, tasks served via API anyway — and first-timers, which is why the lab's default path is hosted.
- Local/rented-GPU LoRA with the open-source training stack — as of 2026, Hugging Face
transformers+PEFT(the LoRA implementation) +TRL(the trainers); these libraries move fast, so substitute your preferred equivalent and check current docs. Full control of base model, template, checkpoints, and knobs; adapters you own as files; costs measured in GPU-hours (a triage-sized LoRA on an 8B: single-digit hours on one modern GPU — rentable by the hour, see the cost callout below). Cons: environment setup is its own skill tax the first time. Right answer for: open-model deployment targets, data that can't transit third parties, and anyone who wants to actually see the machinery (worth doing once even if you'll ship hosted).
Hosted path: no GPU required; you pay per training run. A triage-sized LoRA is a small spend — low tens of dollars, illustrative. Local path: an 8B LoRA fits on a single 24GB GPU; int4/QLoRA shrinks it enough to fit smaller cards. The quantization math you can trust: an 8B model's weights are ≈16GB at fp16, ≈8GB at int8, ≈4GB at int4 (roughly 2 / 1 / 0.5 bytes per parameter), with training overhead on top. Rent a GPU at roughly $0.5–2/hr (illustrative, 2026) for a single-digit-hour run — order of ~$5–15 per run. Both roads are cheap enough that the binding cost is your time, not the compute; check current prices before you budget.
The run protocol (same on both roads)
- Freeze the inputs: dataset version named, base model pinned (exact variant), system prompt fixed to the serving prompt, eval harness ready. A run whose inputs weren't frozen produces a number that means nothing.
- Wire-test (the overfit-20 check on local; a 50-example smoke run on hosted), then launch the real run with default-ish knobs: 2 epochs, default LR, rank 16.
- Watch the curves (or the hosted dashboard's equivalents); note where validation bottoms.
- Evaluate the selected checkpoint on the real eval set — full suite: task accuracy by class, schema adherence rate, escape-case handling, the adversarial slice, the general-capability slice. Compare three columns: raw student + best prompt (the ceiling from Module 1), tuned student, teacher.
- Log the run (config, dataset, scores, artifacts, cost) and decide like an engineer: gap closed enough → Module 5's serving math; format adherence lagging → Module 4's preference pass; task accuracy still short → the answer is almost always in the data, not the knobs — error-analyze the failures by class, patch the curriculum (more of the failing family, better labels), regenerate, retrain. One knob-turn is permitted before you accept that sentence.
Tuned models reliably look better before they are better — outputs adopt your format and tone immediately (that's the easiest thing to learn), which fools quick manual inspection while class accuracy has barely moved. Never grade a fine-tune by vibes or by five samples: the eval suite is the only definition of done, and the by-class breakdown is where the truth lives. (You knew this from RAG; it's truer here, because the cosmetic gains are stronger.)