Semi-Supervised Regression with Heteroscedastic Pseudo-Labels

Deyu Meng (Xi'an Jiaotong University) · Xueqing Sun (Xi'an Jiaotong University) · Renzhen Wang (Xi'an Jiaotong University) · Quanziang Wang (Xi'an Jiaotong University) · Yichen WU (Harvard University) · Xixi Jia (Xidian University)

benchmark datasets bi-level optimization empirical risk minimization error accumulation generalizationheteroscedastic noiseoverfittingpseudo-label reliabilitypseudo-labeling robustness semi-supervised learningsemi-supervised regressionuncertainty estimatesuncertainty-aware framework

Pseudo-labeling is a commonly used paradigm in semi-supervised learning, yet its application to semi-supervised regression (SSR) remains relatively under-explored. Unlike classification, where pseudo-labels are discrete and confidence-based filtering is effective, SSR involves continuous outputs with heteroscedastic noise, making it challenging to assess pseudo-label reliability. As a result, naive pseudo-labeling can lead to error accumulation and overfitting to incorrect labels. To address this, we propose an uncertainty-aware pseudo-labeling framework that dynamically adjusts pseudo-label influence from a bi-level optimization perspective. By jointly minimizing empirical risk over all data and optimizing uncertainty estimates to enhance generalization on labeled data, our method effectively mitigates the impact of unreliable pseudo-labels. We provide theoretical insights and extensive experiments to validate our approach across various benchmark SSR datasets, and the results demonstrate superior robustness and performance compared to existing methods.