weight matrices
Matrices that contain the parameters (weights) of the connections between neurons in a neural network. The learning process adjusts these weights to minimize a loss function during training.
- Accurate and Efficient Low-Rank Model Merging in Core Space
- Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples
- Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
- GAMMA: Gated Multi-hop Message Passing for Homophily-Agnostic Node Representation in GNNs
- Generalization Bounds for Rank-sparse Neural Networks
- GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
- LoRA vs Full Fine-tuning: An Illusion of Equivalence
- Memory by accident: a theory of learning as a byproduct of network stabilization
- PermLLM: Learnable Channel Permutation for N:M Sparse Large Language Models
- Q3R: Quadratic Reweighted Rank Regularizer for Effective Low-Rank Training
- Small Singular Values Matter: A Random Matrix Analysis of Transformer Models