Detecting High-Stakes Interactions with Activation Probes

Ekdeep S Lubana (Goodfire AI) · Alex McKenzie (AE Studio) · Urja Pawar (Bank of New York (BNY)) · Phil Blandfort (Predictably Weird) · William Bankes (UCL) · David Krueger (University of Cambridge) · Dmitrii Krasheninnikov (University of Cambridge)
activation probescomputational savingsdownstream analysisfinetuned monitorshierarchical monitoringhigh-stakes interactionsinitial filtermodel activationsout-of-distribution dataperformance evaluationprobe architecturesresource-aware systemsrobust generalizationsynthetic datasettext-based monitoring

Monitoring is an important aspect of safely deploying Large Language Models (LLMs). This paper examines activation probes for detecting ``high-stakes'' interactions