Capstone: Launch & certify
The staged rollout plan, the demo that shows a failure on purpose, the rubric, and the Certified AI Agent Developer credential.
Customer-facing agents launch in stages, never in leaps — each stage a bigger audience and a decision gate with pre-written criteria:
- Internal dogfood (support team plays customers, 1-2 weeks): the team that will receive handoffs learns the agent by breaking it. Gate: eval floor holds on their transcripts + the reps sign off on the package quality.
- Shadow or fractional traffic (5-10% of real conversations, or agent-drafts-human-sends): real customers, bounded blast radius, daily transcript hours. Gate: the scorecard vs. human baseline — resolution, split CSAT, recontact — for two clean weeks.
- Ramp with the metrics watching (25% → 50% → 100%): each step held long enough for the recontact number to mean something (it lags a week by definition). Any severe incident pauses the ramp automatically — pre-agreed, so nobody negotiates with sunk cost mid-incident.
- Steady state = the operating loop: weekly transcript hour, monthly suite growth, quarterly brief review, the metrics reported in the honest shape (quality floor first, volume second). Launch is the beginning of the product, not the end of the project.
The demo & the rubric
Your final presentation is 10 minutes, live or recorded: two happy paths (fast, grounded, cited), one deliberate failure handled well (an attack refused gracefully or an honest miss routed warmly — showing the failure is the credibility move; every prior course's capstone taught you this and it's truest here), one full handoff with the package shown, and the scorecard with the known-limitations register presented unprompted.
- Design coherence (25%) — brief v2 with a real changelog; one voice across every surface; scope decisions defended.
- Grounding & safety (30%) — citations verified, freshness proven, three layers present, fresh-attack results honest, distress path flawless.
- Handoff & operations (25%) — triggers correct in drills, packages pass the 8-second test, incident path named, off switch accounted for (a plan naming who can throw it and how — pulled and verified if you operate a live toggle).
- Evaluation & launch judgment (20%) — calibrated judge, suite that grew from evidence, metrics defined the honest way, staged plan with real gates.
Passing earns the Certified AI Agent Developer credential (ID format EDOVA-AGT-2026-XXXX, independently verifiable at edova.ai/verify) — attesting that you can put an AI in front of customers and deserve the trust that requires.
Three directions: Agentic AI Systems to give your agent hands (tools, actions, and the guardrail depth acting demands — your handoff triggers become its autonomy dial); LLMOps to run the eval-observe-improve loop at production scale (your suite and judge slot straight in); and AI Product Management for the other half of the job — the metrics, roadmap, and stakeholder craft that decide whether Harbor Helper thrives after launch. If you're building the business case instead of the bot, that's where your story continues.