Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization

Liqiang Nie (Harbin Institute of Technology (Shenzhen)) · Wenqi Liu (Shandong University) · Xuemeng Song (City University of Hong Kong) · Jiaxi Li (University of Georgia) · Yinwei Wei (Shandong University) · Na Zheng (National University of Singapore) · Jianhua Yin (Shandong University)

attention enhancementcomprehensive evaluation direct preference optimizationdirect preference supervisionhallucination mitigationindirect preference supervisionmultimodal large language modelsoptimization objective functionordinal preference learningpreference gap regulationpreference margin consistency losssymmetric multimodal preference optimizationsymmetric preference learningvision-oriented contrastive objectivesvisual understanding enhancement

Direct Preference Optimization (DPO) has emerged as an effective approach for mitigating hallucination in Multimodal Large Language Models (MLLMs). Although existing methods have achieved significant progress by utilizing vision-oriented contrastive objectives for enhancing MLLMs' attention to visual inputs and hence reducing hallucination, they suffer from non-rigorous optimization objective function and indirect preference supervision. To address these limitations, we propose a Symmetric Multimodal Preference Optimization (SymMPO), which conducts symmetric preference learning with direct preference supervision (i.e., response pairs) for visual understanding enhancement, while maintaining rigorous theoretical alignment with standard DPO. In addition to conventional ordinal preference learning, SymMPO introduces a preference margin consistency loss to quantitatively regulate the preference gap between symmetric preference pairs. Comprehensive evaluation across five benchmarks demonstrate SymMPO's superior performance, validating its effectiveness in hallucination mitigation of MLLMs.