More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models

Xin Wang (University of California, Santa Barbara) · James Zou (Stanford) · Zhongxing Xu (Monash University) · Chengzhi Liu (University of California, Santa Barbara) · Qingyue Wei (Stanford University) · Juncheng Wu (UC Santa Cruz) · Yuyin Zhou (UC Santa Cruz) · Sheng Liu (Stanford University)

attention analysisdiagnostic benchmark evaluation frameworksextended reasoning chainshallucination multimodal large language modelsmultimodal math reasoningperception accuracyperceptual reliabilityreasoning abilityreasoning lengthrh-aucrh-benchtraining data domainsvisual grounding

Test-time compute has empowered multimodal large language models to generate extended reasoning chains, yielding strong performance on tasks such as multimodal math reasoning. However, we observe that this improved reasoning ability often comes with increased hallucination: as generations become longer, models tend to drift away from image-grounded content and rely more on language priors. Attention analysis reveals that longer reasoning chains reduce focus on visual inputs, contributing to hallucination. To systematically study this phenomenon, we introduce RH-AUC, a metric that quantifies how a model's perception accuracy changes with reasoning length, enabling evaluation of whether the model preserves visual grounding while reasoning. We also release RH-Bench, a diagnostic benchmark covering diverse multimodal tasks, designed to jointly assess the balance of reasoning ability and hallucination. We find that (i) larger models generally exhibit a better balance between reasoning and perception; (ii) reasoning and perception balance depends more on the types and domains of the training data than its volume. Our findings highlight the need for evaluation frameworks that account for both reasoning quality and perceptual reliability.