Token economics

An AI feature that's excellent and unaffordable doesn't ship — or ships and gets cancelled after the first invoice. Cost is not a finance problem to discover later; it's an engineering metric to design against from the start. Your traces already carry the raw data (Module 3); this module turns it into control.

What drives the bill

• Tokens, in and out, priced separately — output usually costs several times more per token than input, so a verbose feature is expensive twice over. Input volume is often the sneaky one: RAG stuffs chunks, agents accumulate context, and every one rides on every call.
• Calls per request — a single triage is one call; a RAG query is a few; an agent task is many, each carrying the growing history. Cost scales with the interaction, not the request count. A rising calls-per-request average is a cost leak (retries, loops).
• Model choice — frontier models cost multiples of small ones. Using your best model for a job a small one nails is the most common overspend.
• The multipliers — retries, self-consistency votes, judge samples, reflection. Each looks reasonable alone; together they quietly 3× the base. (The silent multipliers, one last time.)

Cost per request, from the trace

You instrumented tokens and cost per span in Module 3. Aggregate it: cost per request = sum of all spans' costs, and report it next to quality on the dashboard. The sentence you're building toward is 'we resolve 94% of cases at $0.34 each, p95 $0.71' — an operable number ($0.34 is an illustrative target here, not a benchmark; yours comes from your own traces). To see how the pieces add up, run one call through the Module 3 formula: ~800 input + 200 output tokens at the illustrative prices ($3.00 / $15.00 per 1M) is 800/1e6×3.00 + 200/1e6×15.00 ≈ $0.005. A request costing $0.34 is therefore many such calls, or one call over far larger context (fat retrieval, long history) — the per-request number is the sum across every span, which is exactly why you total spans rather than eyeball one call. 'The API bill seems high' is not a number you can act on.

The triangle you're actually managing

Quality, cost, and latency trade against each other, always. More reasoning and more retries buy quality with cost and latency. A bigger model buys quality with cost. Aggressive caching buys cost and latency with a little staleness risk. The job is not to maximize quality — it's to hit the quality bar the use case needs at the lowest cost and acceptable latency. A support triage needs to be right and cheap; a legal-review assistant can be slower and pricier if it's more careful. Name your operating point, measure against it, and optimize toward it — the next two lessons are the levers.