Fact-checking AI
A proportionate verification workflow: what to check, where to check it, and how to use a second model as a cheap first auditor.
Verification has a reputation for being the tax that eats the time savings. Done proportionately, it isn't — because you don't check everything, you check the load-bearing claims. A 500-word draft typically rests on three to six of them.
The workflow
- 1Extract the load-bearing claims — the statements which, if wrong, change your conclusion or embarrass you. (Use the (c)-list prompt from the last lesson.)
- 2Match each claim to its authoritative source type: a statistic → the original report or dataset, not a blog quoting it; a law or regulation → the actual text or your counsel; a product capability → the vendor's current docs; a quote → the primary transcript; internal facts → your own systems, always.
- 3Check the strongest claims first. If the most important one fails, you've saved checking the rest of a doomed draft.
- 4Record what you verified. A one-line note ('checked against Q3 10-K, 2026-07-02') turns tomorrow's re-check into a glance.
The second-model audit
A cheap, surprisingly effective layer: paste the draft into a different model (or a fresh chat) and ask it to attack the facts. Different chats don't share your conversation's momentum, so the auditor has no stake in defending the draft.
You are a hostile fact-checker reviewing this text before publication. Your reputation depends on finding errors. For each factual claim: rate plausibility (solid / shaky / suspicious), explain your rating in one line, and say exactly what source would settle it. Be specific — no generic advice. Text: [paste draft]
The auditor prompt catches a lot — but it's a screen, not a proof. Anything still marked 'shaky' that matters goes to a primary source. AI checking AI reduces your workload; it never replaces the final human check on high-stakes claims.
Citations deserve special paranoia
Treat every AI-provided citation as unverified until opened. The failure mode isn't just invented papers — it's real papers that don't say what's claimed, correct-looking URLs that 404, and real authors attached to fictional titles. If your tool has live web search, open its links and read the actual passage. Sixty seconds per citation; careers have ended over skipping it.
Calibrate verification to the audience, not the effort. A Slack message to your team: spot-check. An email to a client: check the (c)-list. A number going into a board deck or a public post: verify every claim to primary sources. The question is never 'how long did the AI take' — it's 'how far does this travel'.