parameter-efficient fine-tuning
This approach refers to methods that enable the fine-tuning of large pre-trained models with minimal adjustment to their parameters, often using techniques such as adapters or low-rank updates. This efficiency is crucial for deploying models in resource-constrained environments while still achieving high performance.
- AR-RAG: Autoregressive Retrieval Augmentation for Image Generation
- Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models
- AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning
- Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
- AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping
- Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
- Bilevel ZOFO: Efficient LLM Fine-Tuning and Meta-Training
- Correlated Low-Rank Adaptation for ConvNets
- CrossSpectra: Exploiting Cross-Layer Smoothness for Parameter-Efficient Fine-Tuning
- Distribution-Aligned Decoding for Efficient LLM Task Adaptation
- Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
- Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
- Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigation
- F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning
- Fantastic Features and Where to Find Them: A Probing Method to combine Features from Multiple Foundation Models
- FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
- Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models
- GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
- GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation
- HALO: Hadamard-Assisted Lower-Precision Optimization for LLMs
- HiMoLE: Towards OOD-Robust LoRA via Hierarchical Mixture of Experts
- LT-Soups: Bridging Head and Tail Classes via Subsampled Model Soups
- Linearization Explains Fine-Tuning in Large Language Models
- Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
- Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation
- Matching Markets Meet LLMs: Algorithmic Reasoning with Ranked Preferences
- Multi-Token Prediction Needs Registers
- Noise-Robustness Through Noise: A Framework combining Asymmetric LoRA with Poisoning MoE
- On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation
- Optimization Inspired Few-Shot Adaptation for Large Language Models
- PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning
- Prompt Tuning Transformers for Data Memorization
- Provable Meta-Learning with Low-Rank Adaptations
- Restoring Pruned Large Language Models via Lost Component Compensation
- Revisiting Semi-Supervised Learning in the Era of Foundation Models
- RidgeLoRA: Matrix Ridge Enhanced Low-Rank Adaptation of Large Language Models
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA
- Seeking and Updating with Live Visual Knowledge
- Towards Robust Parameter-Efficient Fine-Tuning for Federated Learning
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning
- Turning the Tables: Enabling Backward Transfer via Causal-Aware LoRA in Continual Learning
- Uni-LoRA: One Vector is All You Need
- X-Mahalanobis: Transformer Feature Mixing for Reliable OOD Detection