Deploy
Making the agent real: the thin serving layer, the operational runbook, monitoring the metrics that matter, and the staged-autonomy rollout.
A frozen agent in a notebook isn't deployed. Deployment for an acting system is more than a URL — it's the operational shell that lets the agent run against reality without anyone losing sleep. Deliverable 2 is a deployed agent plus the runbook that makes it operable.
The serving layer (thin on purpose)
- An entry point — an endpoint, a queue consumer, a scheduled trigger — that receives tasks and runs the agent loop. Thin: the intelligence is in the agent, the serving layer just feeds and records it.
- The action log and trajectory log persisted — not in memory. These are your audit trail and your debugging record; they outlive the process.
- The kill switch reachable in production — a real control on-call can hit, not a constant in a file you'd have to redeploy. The whole point is stopping a misbehaving agent fast.
- Config, not hardcode — model, thresholds, autonomy dial, budgets. You will tune these against production behavior; make that a config change, not a code deploy.
The runbook (extends your RAG runbook)
One page: agent version + eval scorecard; the tool suite with stakes/reversibility tiers; the guardrail configuration (limits, gates, kill switch location); monitored metrics + alert thresholds; the top 3 likely incidents with first responses (including 'the agent is acting wrong right now → kill switch → query action log → run compensations'); and the escalation owner. If on-call can't stop the agent and reconstruct what it did from this page, it isn't deployed — it's at large.
Monitoring an acting agent (watch the actions, not just the answers)
- Action metrics: write-actions per hour, approval-gate rate and approval ratio (rubber-stamp detector), validator-rejection rate (a spike = something changed — model, prompt, or an attack).
- Health: task success rate on a live-sampled basis, escalation rate, cost per task, turns per task. A rising escalation rate or cost is your early-warning system.
- Safety tripwires that page: any wrong action that reached reality (P0), validator rejections spiking, cost or turn budgets breached, the kill switch fired.
- The weekly loop — sample trajectories (random + low-confidence + any gated), classify with your judge, feed failures to the task suite, feed accepted-but-frequent escalations to whoever can expand the agent's capabilities. The heartbeat you built for RAG, now watching actions.
Never launch an acting agent at full autonomy. Stage the dial: (1) shadow — the agent proposes, a human does everything, you compare; (2) gated — the agent acts on low-stakes/high-confidence, humans approve the rest; (3) supervised auto — most actions auto with heavy sampling; (4) steady state — auto with the weekly loop. Advance a stage only when the metrics from the last one earn it. This is how acting agents reach production without an incident report — and it's exactly the trust ladder every product in this catalog followed.