How ChatGPT is changing the way cybersecurity practitioners look at the potential of AI

Abstraction: ChatGPT dual-use cybersecurity capabilities surprise skeptical security researchers

Key points:

• Security researchers rapidly tested ChatGPT for offensive and defensive tasks: phishing emails, Yara rules, buffer overflow detection, evasion code, and ransomware generation
• Picus Security's Dr. Ozarslan bypassed ChatGPT's content policy by describing ransomware tactics without naming them — got working Swift code to encrypt Office files and exfiltrate via HTTPS
• Generated code tends to be simplistic or buggy; output can be coherent-sounding but factually wrong ("coherent nonsense" per Forrester analyst)
• Ethical filters can be bypassed by framing requests as hypothetical or from a fictional malicious party
• ChatGPT uses reinforcement learning and improves from user interactions, so guardrails could tighten over time
• Practitioners see near-term potential for AI-assisted SOC analyst work, contextualizing alerts and suggesting next steps

Connections: Chatgpt · Openai · Darktrace · Large Language Models · AI Safety · Prompt Engineering

Source: https://www.scmagazine.com/analysis/emerging-technology/how-chatgpt-is-changing-the-way-cybersecurity-practitioners-look-at-the-potential-of-ai