AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
abstentioncomplex problem solvingevaluation frameworkfalse premisesill-posed questionsllm reliabilitymodel scalingoutdated informationreasoning fine-tuningsubjective interpretationssystem promptsystematic evaluationuncertainty reasoningunderspecified queries
For Large Language Models (LLMs) to be reliably deployed in both everyday and high-stakes domains, knowing when not to answer is equally critical as answering correctly.Real-world user queries, which can be underspecified, ill-posed, or fundamentally unanswerable, require LLMs to reason about uncertainty and selectively abstain