Guardrails against hallucination
Harden the agent so it never presents a number it didn't retrieve, and degrades gracefully when it can't answer.
The cardinal sin of an analytics agent is a confident wrong number. You've already built strong defenses; this lesson makes them airtight and adds graceful failure.
The layered defenses (recap + reinforce)
- Prompt: "never claim a figure you did not retrieve from a tool."
- Tools: the only path to a number is
query_metrics, which is always available, so the truthful path is also the easy path. - Layer: every number is governed and tested, so a retrieved number is a correct number.
- Errors as data: wrong metric names bounce back as readable errors, so the model self-corrects instead of guessing.
Degrading gracefully
The agent should say "I can't measure that" clearly, rather than approximate. Two situations to handle:
- Metric doesn't exist ("what's our NPS?") — the model, seeing no matching metric in the catalog, should tell the user it isn't tracked, not estimate.
- Combination is impossible (marketing spend by product) — the layer refuses, the error returns, and the agent explains the limitation instead of forcing a number.
It's tempting to only test the happy path. But the refusals are where trust is won or lost: a business user who catches the agent inventing an NPS will never trust its revenue number again. Verify the "I can't answer that" behavior as carefully as the answers.
Add a limit guard on query_metrics inputs if you expose the agent widely — a sane default limit and a cap on how many rows come back keeps responses fast and costs predictable, without changing correctness.