Black Hat 2023 Review: LLMs Everywhere | DNSFilter

llmcybersecurityblack-hatsecurity-researchnlp

Abstraction: Black Hat 2023 LLM applications in security; best use leverages embeddings not chat interfaces

Key points:

IRonMAN (CyCraft Technology) was standout talk: uses LLM vector representations of Windows command lines to group obfuscated commands across incidents, enabling attribution at machine speed
Key insight: LLMs excel at interpreting natural language text — using numerical embeddings rather than chat interfaces is the highest-value security application
Obfuscation makes equivalent command lines syntactically different; LLM embeddings cluster them by semantic similarity, defeating regex-based detection tools
Weaker talks focused on using ChatGPT chat interfaces to generate exploits — limited value given that ChatGPT gives plausible-but-incorrect Stack Overflow answers 52% of the time
High-value future LLM security applications: assessing risk in source code, web page scripts, and decompiled binaries
Organizations that leverage LLMs for language interpretation (not generation) will extract the most security value

Connections: Dnsfilter · Cycraft Technology · Chatgpt · Large Language Models · Cybersecurity