low-rank adaptation
A technique for fine-tuning large models by introducing low-rank matrices into existing layers, which reduces the number of parameters to be updated while maintaining performance.
- Accurate and Efficient Low-Rank Model Merging in Core Space
- Activated LoRA: Fine-tuned LLMs for Intrinsics
- Adaptive LoRA Experts Allocation and Selection for Federated Fine-Tuning
- AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections
- AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping
- Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
- C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
- Continuous Subspace Optimization for Continual Learning
- Correlated Low-Rank Adaptation for ConvNets
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning
- Dr. RAW: Towards General High-Level Vision from RAW with Efficient Task Conditioning
- Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
- F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning
- FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic
- FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA
- FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
- Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models
- GoRA: Gradient-driven Adaptive Low Rank Adaptation
- GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
- Hawaii: Hierarchical Visual Knowledge Transfer for Efficient Vision-Language Models
- Latent Space Factorization in LoRA
- LoRA vs Full Fine-tuning: An Illusion of Equivalence
- LoRA-EnVar: Parameter-Efficient Hybrid Ensemble Variational Assimilation for Weather Forecasting
- Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
- Low-Rank Head Avatar Personalization with Registers
- LuxDiT: Lighting Estimation with Video Diffusion Transformer
- Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation
- MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
- Multi-Task Vehicle Routing Solver via Mixture of Specialized Experts under State-Decomposable MDP
- Parameter Efficient Fine-tuning via Explained Variance Adaptation
- PoLAR: Polar-Decomposed Low-Rank Adapter Representation
- Ravan: Multi-Head Low-Rank Adaptation for Federated Fine-Tuning
- RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
- RidgeLoRA: Matrix Ridge Enhanced Low-Rank Adaptation of Large Language Models
- Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA
- StelLA: Subspace Learning in Low-rank Adaptation using Stiefel Manifold
- The Primacy of Magnitude in Low-Rank Adaptation
- Towards Robust Parameter-Efficient Fine-Tuning for Federated Learning
- Uni-LoRA: One Vector is All You Need