Jailbreaking Black Box Large Language Models in Twenty Queries

Abstraction: PAIR algorithm uses attacker LLM to jailbreak target LLM in under 20 queries

Key points:

• PAIR (Prompt Automatic Iterative Refinement) uses a separate attacker LLM to generate jailbreaks against any target LLM with only black-box/API access — no model weights needed
• Inspired by social engineering attacks; attacker LLM iteratively refines candidate prompts using in-context learning from the chat history of prior attempts
• Attacker LLM generates a chain-of-thought "improvement" step, explaining its reasoning after each failed attempt
• Often succeeds in fewer than 20 queries — orders of magnitude more efficient than prior methods requiring hundreds of thousands
• Achieves competitive success rates and transferability across GPT-3.5/4, Vicuna, and PaLM-2
• Demonstrates that closed, commercially deployed safety guardrails are vulnerable to automated semantic attacks

Connections: GPT-4 · Palm 2 · LLM Jailbreaking · AI Safety · Red Teaming

Source: https://jailbreaking-llms.github.io/