Hyper-Modality Enhancement for Multimodal Sentiment Analysis with Missing Modalities

Wei Li (National University of Singapore) · Yan Zhuang (University Of Science And Technology Of China) · Minhao Liu (University of Electronic Science and Technology of China) · Wei Bai (Tsinghua University) · Yanru Zhang (University of Electronic Science and Technology of China) · Jiawen Deng (University of Electronic Science and Technology of China) · Fuji Ren (University of Electronic Science and Technology of China)

cross-sample enhancementdata corruptionhyper-modality enhancementincomplete inputsmodality reconstructionmodel performance multimodal sentiment analysis privacy concerns public benchmarks robustness semantic cuessensor failurestate-of-the-art methods training datauncertainty-aware fusion

Multimodal Sentiment Analysis (MSA) aims to infer human emotions by integrating complementary signals from diverse modalities. However, in real-world scenarios, missing modalities are common due to data corruption, sensor failure, or privacy concerns, which can significantly degrade model performance. To tackle this challenge, we propose Hyper-Modality Enhancement (HME), a novel framework that avoids explicit modality reconstruction by enriching each observed modality with semantically relevant cues retrieved from other samples. This cross-sample enhancement reduces reliance on fully observed data during training, making the method better suited to scenarios with inherently incomplete inputs. In addition, we introduce an uncertainty-aware fusion mechanism that adaptively balances original and enriched representations to improve robustness. Extensive experiments on three public benchmarks show that HME consistently outperforms state-of-the-art methods under various missing modality conditions, demonstrating its practicality in real-world MSA applications.