Jailbreak
A prompt or technique designed to bypass an AI system's safety guidelines and get it to produce normally restricted content.
Why this matters
Jailbreaks are the cat and mouse game of AI safety. Users discover tricks to make models ignore their training and do things they're supposed to refuse. Then developers patch those tricks. Then users find new ones. It's been going on since the first chatbots got safety guidelines and it's not stopping anytime soon.
Common techniques include role-playing scenarios, hypothetical framing, asking the AI to pretend it's a different system without restrictions, or crafting prompts that confuse the safety filters. Some jailbreaks are clever social engineering. Others exploit technical quirks in how models process text. The creativity involved is genuinely impressive, even if the intent is often problematic.
The existence of jailbreaks doesn't mean safety measures are useless. They raise the bar significantly. Most casual users won't bother with elaborate workarounds, and the hassle filters out a lot of low-effort misuse. It's similar to door locks. They won't stop a determined burglar, but they prevent most opportunistic problems.
AI companies treat jailbreak research as valuable intelligence. Responsible disclosure of new techniques helps improve defenses. Some security researchers focus specifically on finding and reporting these vulnerabilities. The ongoing battle has actually made models more resilient over time. Perfect defense probably isn't achievable, but good enough defense is a reasonable goal.