Long-tailed Recognition with Model Rebalancing

Feng Liu (University of Melbourne) · Feng Hong (Shanghai Jiao Tong University) · JIAAN LUO (Shanghai Jiao Tong University) · Jiangchao Yao (Shanghai Jiao Tong University) · Xiaofeng Cao (Jilin University) · Qiang Hu (Shanghai Jiaotong University)

class distributiondata augmentationdecoupled trainingfoundation modelsimbalance mitigationlong-tailed recognitionloss rebalancinglow-rank parametermodel capacity model generalizationmodel rebalancingmulti-label recognitionparameter spacesinusoidal reweighting

Long-tailed recognition is ubiquitous and challenging in deep learning and even in the downstream finetuning of foundation models, since the skew class distribution generally prevents the model generalization to the tail classes. Despite the promise of previous methods from the perspectives of data augmentation, loss rebalancing and decoupled training etc., consistent improvement in the broad scenarios like multi-label long-tailed recognition is difficult. In this study, we dive into the essential model capacity impact under long-tailed context, and propose a novel framework, Model Rebalancing (MORE), which mitigates imbalance by directly rebalancing the model's parameter space. Specifically, MORE introduces a low-rank parameter component to mediate the parameter space allocation guided by a tailored loss and sinusoidal reweighting schedule, but without increasing the overall model complexity or inference costs. Extensive experiments on diverse long-tailed benchmarks, spanning multi-class and multi-label tasks, demonstrate that MORE significantly improves generalization, particularly for tail classes, and effectively complements existing imbalance mitigation methods. These results highlight MORE's potential as a robust plug-and-play module in long-tailed settings.