Geometry-Aware Collaborative Multi-Solutions Optimizer for Model Fine-Tuning with Parameter Efficiency

Dinh Phung (Monash University) · Trung Le (University of Washington, Seattle) · Mehrtash Harandi (Monash University) · Van-Anh Nguyen (Monash University) · Ehsan Abbasnejad (Monash University) · Thanh-Toan Do (Monash University)

adaptability bayesian methodscollaborative resultscomputational overhead diverse solutions domain generalization few-shot learning geometric structuregradient flow theoryknowledge preservationparameter updatingperformance enhancementpractical solutionpre-trained foundation models task-specific fine-tuning transfer learning

We propose a framework grounded in gradient flow theory and informed by geometric structure that provides multiple diverse solutions for a given task, ensuring collaborative results that enhance performance and adaptability across different tasks. This framework enables flexibility, allowing for efficient task-specific fine-tuning while preserving the knowledge of the pre-trained foundation models. Extensive experiments across transfer learning, few-shot learning, and domain generalization show that our proposed approach consistently outperforms existing Bayesian methods, delivering strong performance with affordable computational overhead and offering a practical solution by updating only a small subset of parameters.