Back to course overview
Module 5Evaluating output 20 min

Lab: Output review

A graded exercise: find the planted errors in a polished AI report, then run the full verification workflow on your own workflow's output.

Two parts: first you'll hunt planted errors in a controlled setting, then you'll turn the skills on your own Module 3 workflow. This lab is the difference between having read about verification and having done it.

Part 1 — The seeded report

Generate your practice document with this prompt:

Prompt to try

Write a confident, polished 400-word market brief on [an industry you know well] for an internal audience. Deliberately include: two accurate widely-known facts, two subtly wrong 'facts' (plausible but incorrect), one invented statistic with a fake source, and one unfalsifiable filler claim. Do NOT mark which is which. Number the paragraphs.

  1. 1Read the brief once at normal speed — the speed you'd really read at on a busy day. Mark what feels off.
  2. 2Now run the 30-second triage from lesson one: claims vs. transformations, then the (c)-list prompt.
  3. 3Fact-check your (c)-list using real sources (or the auditor prompt in a fresh chat, then sources for anything shaky).
  4. 4Ask the original chat to reveal the plants. Score yourself: caught / missed / false alarms.

A typical run looks like this: you catch the invented statistic, miss one subtle wrong fact, and false-alarm on one accurate fact. That pattern is the lesson: expect to catch only around half the planted errors unaided — the workflow closes the gap.

Part 2 — Audit your own workflow

  1. 1Take the most recent real output from your Module 3 workflow.
  2. 2Run the full treatment: triage → (c)-list → verify the top three claims to primary sources → second-model audit.
  3. 3Write down: what you found, how long it took, and where in the workflow a checkpoint would have caught it earlier.
  4. 4Update your workflow one-pager with the checkpoint and a spot-audit cadence (every Nth run).

Problem set 5

Three AI outputs at three stake levels. For each, design the proportionate review: what you'd check, against what source, and what you'd deliberately skip. The skill being practiced is proportionality, not maximal paranoia — the right answer differs sharply across the three.

  1. A recap of yesterday's standup, posted to your team's Slack channel.
  2. An email to a client confirming revised delivery dates and a small credit.
  3. A market-size figure with a cited source, going on a slide for the board.