Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs

Xiaofeng Zhang (Harbin Institute of Technology) · Yifan Shen (Mohamed bin Zayed University of Artificial Intelligence) · Yuanzhe Liu (Rensselaer Polytechnic Institute) · Ismini Lourentzou (University of Illinois Urbana-Champaign) · James Rehg (University of Illinois at Urbana-Champaign) · Xu Cao (University of Illinois at Urbana-Champaign & PediaMed AI) · Jingyuan Zhu (University of Pennsylvania) · Yixiao He (Beijing University of Posts and Telecommunications) · Wenming Ye (Google)
average accuracycompetitive performancedescriptive groundingdirect preference optimizationlogical coherencelogical reasoninglong chain-of-thoughtmulti-model monte carlo tree searchsegment-specific preference granularityspatial groundingspatial reasoningspatial reward mechanismspatialrgpt-benchvision-language modelsvisual consistency

Current Vision-Language Models (VLMs) struggle with fine-grained spatial reasoning, particularly when multi-step logic and precise spatial alignment are required. In this work, we introduce SpatialReasoner-R1, a vision-language reasoning model designed to address these limitations. To construct high-quality supervision for spatial reasoning, we design a Multi-Model Monte Carlo Tree Search (M3CTS) method that generates diverse, logically consistent Long Chain-of-Thought (LongCoT) reasoning trajectories. In addition, we propose a fine-grained Direct Preference Optimization (fDPO) method that introduces segment-specific preference granularity for descriptive grounding and logical reasoning, guided by a spatial reward mechanism that evaluates candidate responses based on visual consistency, spatial grounding, and logical coherence. Experimental results demonstrate that fDPO achieves relative performance gains of 4.1% and 9.0% over standard DPO on spatial quality and spatial quantity tasks, respectively. SpatialReasoner-R1, trained with fDPO, sets a new SoTA on SpatialRGPT-Bench, outperforming the strongest baseline by 9.8% in average accuracy, while maintaining competitive performance on general vision-language tasks.