Training basics (what actually matters)
Loss curves you can read, the three hyperparameters worth touching, overfitting as the default failure, and catastrophic forgetting as the tax on narrow training.
Training exposes a hundred knobs; experienced practitioners touch about three and watch two charts. This lesson is the watching-and-touching guide — deliberately practical, with pointers for the theory-hungry.
- Loss, and how to read its two curves. Loss ≈ 'how surprised is the model by the training outputs' — training pushes it down. You watch training loss and validation loss together, and their relationship is the whole diagnosis: both falling = learning; training falling while validation rises = overfitting (the model is memorizing your examples instead of your task — the default failure of small-dataset fine-tuning, expect to meet it); both flat from the start = something's mis-wired (learning rate, template, data) — stop and check, don't wait it out.
- The three knobs: epochs, learning rate, LoRA rank. Epochs (passes over the data): 1-3 for small SFT sets; more epochs on a small dataset is the overfitting express. Learning rate (step size): the tooling's default for your setup is right more often than your intuition; if outputs degrade into repetition or chaos, it was too high — go down, not up. LoRA rank (adapter capacity): 8-16 for format/style tasks, 32-64 when the task needs more room; higher rank = more capacity = more overfitting risk on small data. Change one knob per run and log every run (config, dataset version, scores) — the experiment log is the difference between tuning and thrashing.
- Catastrophic forgetting — narrowness has a price. Train hard on triage and the model gets worse at everything else: general instruction-following, safety behaviors, chat coherence. For a single-purpose deployment this can be acceptable — but 'acceptable' is a measurement, not a hope: your eval suite carries a small general-capability slice (a dozen off-task prompts scored for basic sanity) precisely to watch this. A triage model that's forgotten how to refuse harmful requests is not a triage model you can ship.
Checkpoints and the selection discipline
Training saves checkpoints (the model at intervals), and the final checkpoint is frequently not the best one — validation loss often bottoms mid-run and rises after (overfitting's onset). Selection rule: pick the checkpoint at the validation minimum, then confirm on the real eval set — because validation loss is a proxy, and the eval set is the truth. Best-checkpoint-by-eval is the model you carry forward; everything else is deleted with gratitude.
Before your real run, do the classic wiring test: train on 20 examples for many epochs and confirm the model can nail those 20 (training loss near zero, outputs verbatim-ish). If it can't overfit a tiny set, something upstream is broken — template mismatch, data mangling, wrong target — and no full run will fix it. On the local path this is five minutes of GPU; on the hosted path, where you can't force a 20-example run cheaply, the equivalent sanity check is a small smoke run (~50 examples) that confirms the pipeline accepts your file, trains, and returns a model that moved. Either way it's saved more debugging weeks across the industry than any other single habit.