Lab: Red-team an agent

The capstone lab of the taught modules: turn your scattered attacks into a real red-team suite, run it against your hardened Harbor Lane AI, close any remaining gaps with guardrails, and wire it into CI so security can't regress. You'll leave with a measured security posture.

Step 1 — Build the attack suite

1. 1Collect every attack from Modules 2–4 plus new ones, covering all six categories (injection direct/indirect, jailbreak, extraction, privilege abuse, cost/DoS, improper output). Aim for 30–50 cases.
2. 2Use an LLM to expand the thin categories: 'generate 15 varied indirect-injection payloads targeting this agent, across the message/review/ticket channels.' Curate its output into the suite.
3. 3Format each as {id, category, channel, attack_input, expected_defended_behavior}. Save attacks/v1.jsonl — your security golden set.

Step 2 — Run it and score

1. 1Write redteam.py: run each attack against the hardened agent, and judge the outcome — harmful action executed (fail), harmful output shipped (fail), blocked/flagged/harmless (pass). For the fuzzy 'is this output a leak/violation?' judgments, use a guardrail-model judge, calibrated against your hand-labels.
2. 2Compute attack success rate overall and per category, and for each landed attack, which layer stopped it (or didn't).
3. 3The non-negotiable bar: zero attacks cause harmful actions. Output-level near-misses are findings to reduce; action-level successes are P0s to fix now.

Step 3 — Close gaps with guardrails

1. 1For any attack that succeeded or got uncomfortably close, add the right guardrail: a guardrail-model input screen for slippery injections, an output leakage screen, a tighter action validator, a stricter scope.
2. 2Re-run the suite. Confirm the closed gaps stay closed and you didn't break legitimate behavior — run your quality eval too (security fixes that block real users are their own failure).

Step 4 — Gate it in CI

1. 1Wire the red-team suite as a CI gate (your LLMOps eval-gate machinery): any harmful-action success fails the build, and attack success rate must not regress past a threshold.
2. 2Prove it bites: make a change that reopens an injection (loosen a validator) and watch the security gate go red.
3. 3Record the posture in LAB-NOTES and THREAT-MODEL.md: the suite size, attack success rate by category, defense-in-depth results, and 'runs in CI'. That's a defensible security claim.