lora
LoRA (Low-Rank Adaptation) is a technique used to reduce the number of parameters that need to be fine-tuned in large pre-trained models by introducing low-rank matrices for the adaptation process, making it more efficient and accessible for resource-constrained environments.
- AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning
- Block-Diagonal LoRA for Eliminating Communication Overhead in Tensor Parallel LoRA Serving
- EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction
- FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic
- HiMoLE: Towards OOD-Robust LoRA via Hierarchical Mixture of Experts
- LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossing
- MolVision: Molecular Property Prediction with Vision Language Models
- Preventing Shortcuts in Adapter Training via Providing the Shortcuts
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment
- Provable Meta-Learning with Low-Rank Adaptations
- Restoring Pruned Large Language Models via Lost Component Compensation
- SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification
- Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation is Wasteful