Back to course overview
Module 2Planning & reasoning 20 min

Lab: Planner design

Upgrade the resolution agent to plan-as-artifact with steered reasoning and reflection rules — measured on turns, tokens, and completeness.

The resolution agent learns to plan. You'll add the plan-as-artifact pattern, tune reasoning depth, wire the two cheap reflections — and measure whether any of it earned its tokens, because 'feels smarter' is not a metric.

Step 1 — Baseline the hard case

  1. 1Add ticket 5 to your world: a multi-request mess — damaged item + wrong address on a second order + a question the docs don't cover. The kind of ticket that breaks improvisers.
  2. 2Run v0.1 on it three times. Record per run: turns, total tokens, and a completeness check by hand (which of the three requests got handled?). This is your baseline table.

Step 2 — v0.2: the planner

  1. 1System prompt: first action on any ticket = emit a numbered plan (milestone granularity); each turn thereafter names the current step; revisions must be explicit ('revising plan: step 3 obsolete because…'). Cap revisions at 2.
  2. 2Reasoning dial: one sentence before reads; full facts-policy-amount reasoning before writes; 'briefly' everywhere.
  3. 3Reflection rules: verify-after-act on every update_ticket; the pre-finalize checklist from the last lesson.

Step 3 — Measure the same way

  1. 1Run v0.2 on ticket 5 three times. Same table: turns, tokens, completeness.
  2. 2Also re-run tickets 1-3 (the easy ones): planning overhead should be small there — if the simple where's-my-order ticket now takes 9 turns of ceremony, your plan granularity is too fine.
  3. 3Read one v0.2 trajectory end to end. Mark: did the plan survive contact with the missing-docs surprise? Was the revision explicit? Did the checklist catch anything?

Step 4 — The verdict table

Write the comparison into LAB-NOTES.md: v0.1 vs v0.2 on easy and hard tickets — turns, tokens, completeness, plus one sentence of judgment. The expected shape: planning costs a little on easy tasks and pays a lot on hard ones; reflection lifts completeness from ~2/3 requests to 3/3. If your numbers disagree, investigate before believing either the lesson or the run — that instinct is the course.

Problem set 2

In the workbook: three trajectories from the same task — a wanderer (no plan, 22 turns), a rigid planner (executed a stale plan into a wall), and a healthy plan-revise-complete run. Diagnose each in two sentences and prescribe the one prompt change that fixes each failure.