The reliability gap
Why AI features that demo perfectly fall apart in production — the four sources of the reliability gap, and why traditional ops doesn't close it.
You've built impressive AI systems across this track: a triage assistant, a RAG document-Q&A engine, an agent that resolves cases. Each worked in the labs. This course is about the distance between 'worked in the labs' and 'runs reliably for real users, unattended, at 3am' — a distance every team underestimates exactly once.
The demo-to-production gap
Traditional software has a small, well-understood gap: it did the same thing in the demo as in production, because software is deterministic. AI features have a reliability gap that's wider and stranger, and it comes from four sources you've met individually — now you'll manage them together:
- Non-determinism. The same input can produce different outputs run to run. A feature that passed your five manual checks fails the sixth identical call. Traditional testing assumes repeatability; here there is none.
- The long tail of inputs. Your demo used inputs you chose. Production sends inputs you never imagined — the empty message, the 40-page paste, the question in Portuguese, the injection attempt. The tail is where AI features break, and the tail is infinite.
- Silent degradation. Software fails loudly — a stack trace, a 500. AI fails fluently: a confident wrong answer, a plausible hallucination, a subtly-off classification. Nothing crashes. The failure looks exactly like success until a customer or an executive finds it.
- Drift you didn't cause. The model provider updates the model. Your corpus grows. User questions shift. The system you tested is not the system running today, and nothing in your code changed to tell you.
Why traditional ops isn't enough
Your uptime dashboard says 100%. Your error rate is 0.01%. Every metric is green — and the AI feature is quietly giving wrong answers to a third of a category of questions. **Traditional observability measures whether the system responded; LLMOps measures whether it responded correctly.** That gap — green infrastructure over a broken feature — is the specific problem this discipline exists to solve.
Across four courses you built the Harbor Lane support AI. This course operates it. You'll treat the triage assistant, HarborDocs, and the resolution agent as one production system and give it what production demands: an eval harness that runs on every change, tracing you can debug from, versioned prompts, a cost budget you enforce, and a CI gate that stops a regression before it ships. By the capstone you'll have an LLMOps stack you can wrap around any AI feature — including one of your own.
This course assumes the eval and guardrail instincts from Prompt Engineering, RAG, and Agentic AI — golden sets, judges, validators, routing. LLMOps doesn't teach new AI techniques; it teaches the operational scaffolding that makes them survive production. Bring one of your prior projects (no mock is shipped — if you have none, you'll stub the feature entry point in the first lab) and an Anthropic API key.