Can AI really be protected from text-based attacks? | TechCrunch

Abstraction: LLM prompt injection attacks are low-barrier and currently unpreventable

Key points:

• Prompt injection compared to privilege escalation attacks; users tricked Bing Chat into revealing hidden system prompts and producing toxic content
• Meta BlenderBot and OpenAI ChatGPT have both been prompted to reveal internal workings or produce offensive material
• No reliable way to detect injection because LLMs don't know when injection occurred — models cannot emit "this chain caused injection"
• Attacks require no technical skill; an articulate person without coding knowledge can elicit undesirable behavior
• Defenses include input/output content filters and RLHF alignment; Microsoft deployed Bing Chat mitigations the same week
• Experts expect an arms race analogous to cybersecurity; bug bounty programs proposed to incentivize responsible disclosure

Connections: Microsoft · Openai · Chatgpt · Prompt Injection · Adversarial Machine Learning · Large Language Models

Source: https://techcrunch.com/2023/02/24/can-language-models-really-be-protected-from-text-based-attacks/