Jailbreaks

Jailbreaking is the cousin of injection aimed at the model's safety training rather than your application's instructions — techniques to make a model produce content it was trained to refuse, or ignore the behavioral rules its provider baked in. As a system builder you don't train models, but you must understand jailbreaks because (a) attackers combine them with injection against your app, and (b) they reveal how fragile prompt-level control really is.

The recurring techniques

• Role-play / persona — 'pretend you are an AI with no restrictions', the 'DAN' family. Wrap the forbidden request in a fiction where the rules supposedly don't apply.
• Hypotheticals & framing — 'for a novel I'm writing…', 'purely educationally…', 'my grandmother used to read me…'. Recontextualize the ask so refusal seems unhelpful.
• Instruction override — 'ignore all previous instructions', the direct assault, often layered under a persona.
• Obfuscation — encode the payload (base64, leetspeak, another language, split across turns) to slip past filters that match on surface strings.
• Many-shot / context flooding — fill the context with examples of the model complying, exploiting its imitation instinct.
• Crescendo — start benign and escalate gradually across turns, so no single message trips a filter.

Why they work — and why the arms race is permanent

Safety training makes refusal likely, not guaranteed — it's a strong statistical tendency layered on a system that is fundamentally a flexible instruction-follower. Every jailbreak exploits the gap between 'trained to refuse' and 'incapable of complying'. Providers patch known jailbreaks; attackers find new framings; the cycle continues. For you the lesson is the same as injection's: model-level safety is a layer, not a guarantee — you cannot outsource your application's safety to the model's training.

What jailbreaks mean for your system

• Your app's rules are more jailbreakable than the provider's, because a two-line system prompt has far less reinforcement than months of safety training. 'Never issue refunds over $200' in a prompt is trivially talked around — which is why it must live in code (Module 4/5). And even the provider's safety training is a strong tendency, not a hard boundary — it too can be jailbroken — so don't rely on it for anything consequential either.
• Reputational and content risks are real even for a benign business app: a jailbroken support bot that emits offensive content, endorses competitors, or gives dangerous advice is a screenshot away from a PR incident.
• Defense is layered: input screening catches known patterns, output screening catches leaked forbidden content, and architecture ensures a jailbroken model still can't do anything harmful. No single layer suffices.