Lab: Planner design
Upgrade the resolution agent to plan-as-artifact with steered reasoning and reflection rules — measured on turns, tokens, and completeness.
The resolution agent learns to plan. You'll add the plan-as-artifact pattern, tune reasoning depth, wire the two cheap reflections — and measure whether any of it earned its tokens, because 'feels smarter' is not a metric.
Step 1 — Baseline the hard case
- 1Add ticket 5 to your world: a multi-request mess — damaged item + wrong address on a second order + a question the docs don't cover. The kind of ticket that breaks improvisers.
- 2Run v0.1 on it three times. Record per run: turns, total tokens, and a completeness check by hand (which of the three requests got handled?). This is your baseline table.
Step 2 — v0.2: the planner
- 1System prompt: first action on any ticket = emit a numbered plan (milestone granularity); each turn thereafter names the current step; revisions must be explicit ('revising plan: step 3 obsolete because…'). Cap revisions at 2.
- 2Reasoning dial: one sentence before reads; full facts-policy-amount reasoning before writes; 'briefly' everywhere.
- 3Reflection rules: verify-after-act on every
update_ticket; the pre-finalize checklist from the last lesson.
Step 3 — Measure the same way
- 1Run v0.2 on ticket 5 three times. Same table: turns, tokens, completeness.
- 2Also re-run tickets 1-3 (the easy ones): planning overhead should be small there — if the simple where's-my-order ticket now takes 9 turns of ceremony, your plan granularity is too fine.
- 3Read one v0.2 trajectory end to end. Mark: did the plan survive contact with the missing-docs surprise? Was the revision explicit? Did the checklist catch anything?
Step 4 — The verdict table
Write the comparison into LAB-NOTES.md: v0.1 vs v0.2 on easy and hard tickets — turns, tokens, completeness, plus one sentence of judgment. The expected shape: planning costs a little on easy tasks and pays a lot on hard ones; reflection lifts completeness from ~2/3 requests to 3/3. If your numbers disagree, investigate before believing either the lesson or the run — that instinct is the course.
In the workbook: three trajectories from the same task — a wanderer (no plan, 22 turns), a rigid planner (executed a stale plan into a wall), and a healthy plan-revise-complete run. Diagnose each in two sentences and prescribe the one prompt change that fixes each failure.