Jailbreak Detection

Jailbreak detection identifies attempts to bypass AI safety guardrails or content policies. What are jailbreaks?: Prompts designed to make models ignore safety training, generate harmful content, or behave against guidelines. "DAN" prompts, roleplay exploits, encoded instructions. Detection approaches: Classifier-based: Train models to recognize jailbreak patterns, flag suspicious inputs. Rule-based: Detect known attack patterns, prompt templates, suspicious formatting. Behavioral: Monitor for policy-violating outputs, unusual response patterns. LLM-as-detector: Use another model to analyze if input is adversarial. Signals: Roleplay setups, instruction override attempts, encoded/obfuscated text, hypothetical framings, multi-turn escalation. Response options: Block request, refuse gracefully, alert for review, log for analysis. Arms race: New jailbreaks constantly discovered, detection must evolve. Implementation: Input filter before main model, output filter after, or both. Tools: Rebuff, NeMo Guardrails, custom classifiers. Trade-offs: False positives frustrate users, false negatives allow harm. Continuous monitoring and updating essential for production safety.