Learn and Ensemble Bridge Adapters for Multi-domain Task Incremental Learning

Xin Liu (Peng Cheng Laboratory) · Zhen Cui (Beijing Normal University) · Yide Qiu (Nanjing University of Science and Technology) · Ziqi Gu (Nanjing University of Science and Technology) · Chunyan Xu (Nanjing University of Science and Technology) · Wenxuan Fang (Nanjing University of Science and Technology)

catastrophic forgettingcontinuous-domain bridge adaptationdiffusion model domain-specific expertise generalization capabilitieshistorical adaptersknowledge transferlearnable weightslearning and ensembling bridge adapterslifelong learningmemory consolidationmulti-domain task incremental learningprogressive knowledge ensembleschrödinger bridgesimilarity-based selection

Multi-domain task incremental learning (MTIL) demands models to master domain-specific expertise while preserving generalization capabilities. Inspired by human lifelong learning, which relies on revisiting, aligning, and integrating past experiences, we propose a Learning and Ensembling Bridge Adapters (LEBA) framework. To facilitate cohesive knowledge transfer across domains, specifically, we propose a continuous-domain bridge adaptation module, leveraging the distribution transfer capabilities of Schrödinger bridge for stable progressive learning. To strengthen memory consolidation, we further propose a progressive knowledge ensemble strategy that revisits past task representations via a diffusion model and dynamically integrates historical adapters. For efficiency, LEBA maintains a compact adapter pool through similarity-based selection and employs learnable weights to align replayed samples with current task semantics. Together, these components effectively mitigate catastrophic forgetting and enhance generalization across tasks. Extensive experiments across multiple benchmarks validate the effectiveness and superiority of LEBA over state-of-the-art methods.