Assess
Run a real security assessment on a system of your choice: scope, threat model, and an attack campaign that produces prioritized, evidenced findings.
Your capstone is a full security assessment of an AI system — the deliverable a security engineer produces for real: find the vulnerabilities, prove them, prioritize them, then (next lessons) fix and report. Ideally you assess a system you built in a prior course; the provided vulnerable stack is the fallback. This week: assess.
Choosing and scoping the target
- Best: your own prior project — a RAG app or agent you built, now put under adversarial scrutiny. Assessing your own work is uncomfortable and instructive in equal measure.
- Scope explicitly: what's in bounds (the app, its tools, its data flows) and out (the model provider's infrastructure, the underlying cloud — you assess your system, not theirs). A written scope is the first mark of a professional assessment.
- Get authorization — even for your own system, note that you're authorized to test it. Against anything you don't own, authorization isn't a formality; it's the line between security research and a crime.
The assessment method
- 1Threat model (Module 1): data-flow diagram, input channels, trust boundaries, threats mapped to OWASP, ranked by risk. The map for everything after.
- 2Attack campaign (Modules 2–5): systematically probe the top-ranked threats. Injection (direct + indirect), leakage, privilege abuse, jailbreaks — run your red-team categories against the target.
- 3Evidence every finding: the attack input, the reproduction steps, the observed impact (with the trace/log), and the OWASP category. An unreproducible finding isn't a finding; a finding without evidence is an opinion.
- 4Score each by likelihood × impact (blast radius). Critical / High / Medium / Low. The ranked findings list is the spine of the report.
I'm assessing this AI system: [describe architecture, tools, data]. Help me plan the attack campaign: given its highest-risk threats, list the specific attacks I should attempt in priority order, the channel for each, and what a successful exploit would prove. Then flag any attack surface I'm likely underestimating.
A planning aid, not a substitute for hands-on attacking — the model is good at systematic coverage, you're responsible for actually landing (or failing to land) each attack and evidencing the result.
The output of Assess week is a prioritized, evidenced findings list — each with a reproduction, an impact, and a severity. That artifact alone is more than most AI systems ever receive. Resist the urge to fix as you go; a clean assessment first, then a deliberate hardening pass, is how professionals avoid missing the forest for one interesting tree.