Back to course overview
Module 6Evaluation 11 min

Cost per task

The economics that decide whether an agent ships: what drives per-task cost, the quality/cost/latency triangle, and optimizing without gutting reliability.

An agent that resolves cases perfectly for $6 each may be a worse product than one that resolves 95% of them for $0.30 and escalates the rest. Agent economics are unforgiving because the costs multiply: every task is many model calls, each carrying accumulated context. Cost per task is not a footnote — it's often the metric that decides whether the agent ships at all.

Where the money goes

  • Turns × context. The dominant driver: each turn re-sends the growing message list. A 15-turn task with fat context can cost 10× a 5-turn task — and context grows within a task, so long trajectories are super-linear. This is why Module 4's compaction is also cost control.
  • Reasoning tokens. Un-steered verbose reasoning between every call multiplies output cost. 'Briefly' is a budget line.
  • Tool-call overhead. Each tool result adds context that rides along for the rest of the task. Unshaped results are a cost leak that compounds per remaining turn.
  • Retries, reflection, self-consistency. Every 'run it again to be safe' multiplies the base — invisibly, because each looks reasonable alone (the silent multipliers from RAG, at agent scale).

The triangle you're actually optimizing

Quality, cost, and latency trade against each other, and pretending otherwise is how agents get cancelled after the pilot. More reasoning and more reflection buy quality with cost and latency. A bigger model buys quality with cost. Aggressive compaction buys cost with a little quality risk. Optimize for the operating point the use case needs, not the maximum of any one corner — and make the trade explicit, measured on your task suite, not guessed.

  • Model tiering — the biggest lever: a small model for routine cases, escalate to your best model only for hard or low-confidence ones. Route by case difficulty (a cheap classifier or the confidence signals you already compute).
  • Cheaper sub-calls — query rewriting, summarization for compaction, and simple judge passes don't need your flagship model.
  • Cap the loop — turn budgets aren't just safety; a task that needs 20 turns is a task to escalate, not to keep paying for.
  • Cache the cacheable — repeated tasks, stable system prompts (prompt caching), resolved-before cases. RAG's caching lessons transfer directly.
Compute cost per task from day one

From your action + trajectory logs: tokens × prices, summed per task, reported next to success rate. 'We resolve 94% of cases at $0.34 each, p95 $0.71, escalating 6%' is a business case. 'The agent works great' is a science-fair project. The teams whose agents survive contact with finance are the ones who could always answer 'what does one resolution cost?'