Lab: Fine-tune the student

The course's centerpiece lab: dataset v1.0 meets the 8B student. Follow the protocol; the deliverable is not 'a model' but a logged, evaluated, error-analyzed run — plus one improvement loop, because the first run is never the last.

The training-file shape. SFT data ships as JSONL — one JSON object per line, messages format (current across hosted APIs and the open stack):

{"messages": [{"role": "system", "content": "You are a support-email triage classifier. Return JSON."}, {"role": "user", "content": "Subject: Where is my order?\nMy package hasn't arrived and it's been two weeks."}, {"role": "assistant", "content": "{\"category\": \"order_status\", \"urgency\": \"medium\", \"extraction\": {\"order_id\": null}}"}]}
{"messages": [{"role": "system", "content": "You are a support-email triage classifier. Return JSON."}, {"role": "user", "content": "Subject: Refund please\nI was double-charged on order A-4471 and want my money back."}, {"role": "assistant", "content": "{\"category\": \"refund_request\", \"urgency\": \"high\", \"extraction\": {\"order_id\": \"A-4471\"}}"}]}

A minimal LoRA/SFT skeleton for the local path — a reference shape, not guaranteed-runnable. The training APIs move fast; treat every argument name as something to confirm against current TRL/PEFT docs:

# Reference shape, NOT guaranteed-runnable. Training APIs move fast.
# Pinned for orientation: transformers 4.x, peft 0.x, trl 0.x (2026).
# Check current TRL/PEFT docs before relying on any argument name below.
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

base = "meta-llama/Llama-3.1-8B-Instruct"  # your 8B student
model = AutoModelForCausalLM.from_pretrained(base)
tokenizer = AutoTokenizer.from_pretrained(base)

lora = LoraConfig(
    r=16,                 # rank: adapter bottleneck (8-16 for format/style)
    lora_alpha=32,        # scaling, commonly 2x rank
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_dropout=0.05,
    task_type="CAUSAL_LM",
)

# Memory-relevant knobs are noted inline; defaults suit a triage-sized run.
config = SFTConfig(
    output_dir="./triage-student",
    per_device_train_batch_size=2,   # raise until you hit GPU memory
    gradient_accumulation_steps=8,   # effective batch = batch * accum
    num_train_epochs=2,              # 1-3 for small SFT sets
    learning_rate=2e-4,              # typical LoRA LR; go down if it degrades
    max_length=1024,                 # cap to your p95 input; smaller = cheaper
)

trainer = SFTTrainer(
    model=model,
    args=config,
    train_dataset=train_dataset,     # your loaded JSONL split
    eval_dataset=eval_dataset,
    peft_config=lora,
)
trainer.train()

1. 1Choose your road and freeze inputs: hosted (default) or local LoRA (if you have the GPU access and appetite). Pin the base model, name dataset v1.0, fix the serving system prompt, confirm the eval harness runs end to end on 5 cases.
2. 2Wire-test, then train: the overfit/smoke check first — on the record in your run log — then the real run (2 epochs, defaults, rank 16 if exposed). While it runs, pre-register your predictions in the log: expected overall score, which classes you think will lag. (Pre-registration makes your error analysis honest — you'll see what surprised you, which is where learning lives.)
3. 3Select and evaluate: best checkpoint by validation → full eval suite → the three-column table (ceiling / tuned / teacher) with by-class breakdown, schema adherence, escapes, adversarial, general-capability slice. Typical first-run shape for this project: overall jumps most of the way to teacher (~90-94% vs. teacher ~97%), schema adherence way up, but 1-2 classes still lagging and the adversarial slice soft.
4. 4Error-analyze like Module 2 taught you to fear: pull every eval failure, cluster by class and cause (label ambiguity? curriculum under-weight? genuinely hard?). Write the one-page findings note: which failures are data-fixable (most), which are capacity (few, at this task size), which are eval-set bugs (always some — fix the eval, log the fix).
5. 5One improvement loop, data-first: patch the curriculum for the worst class (more examples, cleaner labels — regenerate via the Module 2 pipeline, bump to dataset v1.1 with manifest), retrain with identical knobs, re-evaluate. The expected result — a class-targeted gain from a data-targeted fix — is the course's central lesson landing empirically: in adaptation, the data is the steering wheel; the knobs are the seat adjustment.