Incident response

Controls reduce incidents; they never eliminate them. A mature governance program plans for the day an AI system fails publicly, harmfully, or unlawfully — because that day comes, and the organizations that weather it are the ones that rehearsed. AI incident response is a distinct competency, because AI incidents differ from security incidents: the harm is often statistical and slow (a bias that accumulated over months), fluent (a wrong output that looked right), and hard to scope (how many decisions did the broken model affect?).

What counts as an AI incident

• Harm to people — a discriminatory pattern in the HR tool's rejections; a support agent that gave dangerous advice; a wrongful automated denial.
• Security breach — a successful injection that moved money or exfiltrated data (the security course's failures, realized).
• Compliance failure — operating a high-risk system without a required control; a data-protection violation.
• Systemic malfunction — model drift or a bad deploy degrading a consequential system at scale before detection.
• The defining question: an AI incident is any event where the system caused, or credibly could have caused, harm or non-compliance — and the 'could have' near-misses are the free lessons you should treat almost as seriously as the real ones.

The response lifecycle

1. Detect & triage — the detective controls surface it (monitoring, a complaint, an audit). Assess severity and scope fast: what system, what harm, how many affected?
2. Contain — stop the bleeding with the corrective controls: kill switch, rollback, route to humans, disable the tool. The engineering work makes this seconds, not days — which is why it's a governance requirement, not just an engineering nicety.
3. Investigate — the audit logs and lineage answer 'what happened, why, and who/what was affected?' Evidence you designed in is what makes a real investigation possible.
4. Remediate & recover — fix the root cause, and make affected people whole (review and reverse wrongful decisions, notify, compensate). AI incidents have victims, not just bugs.
5. Notify — regulators and affected parties, where required. GDPR breach-notification timelines, EU AI Act serious-incident reporting for high-risk systems, and sectoral duties can be tight (hours to days). Knowing your notification obligations before the incident is the difference between compliant and penalized.
6. Learn — post-incident review; the failure becomes a new control, a new eval case, an inventory update. The governance flywheel, closed.

Build it: Meridian's incident plan

1. 1Define and triage. Write Meridian's incident definitions — what counts as an AI incident: a harmful output that reached a person, a control failure, drift past a monitored threshold, an unannounced vendor model change — and a severity scale that triages each in minutes, not meetings.
2. 2Name the roles and the kill-switch authority. Who leads the response, who loops in legal and communications, and — a named person, not a committee — who may stop each system. For the HR tool: who can halt screening today, and what happens to the candidate pipeline while it's down.
3. 3Draw the notification decision tree, with deadlines. Internal escalation first, then the external branches: affected individuals, and regulators where required (GDPR breach windows, EU AI Act serious-incident reporting for high-risk systems, sectoral duties). Put the deadline on every branch — knowing your notification clocks before the incident is most of the plan.
4. 4Script one tabletop and run it. Scenario: the bias monitor finds the HR tool scoring one demographic group lower. Walk the plan end to end — detection, severity call, kill-switch decision, scoping (how many candidates affected, since when?), notification branches, remediation of the affected decisions. Save INCIDENT-PLAN.md: definitions and severity triage, roles and kill-switch authority, the decision tree, and the tabletop script — the next lesson's roadmap and the capstone both reference it.