spectral properties
Spectral properties pertain to the characteristics of matrices (e.g., weight matrices in neural networks) associated with eigenvalues and eigenvectors. They provide insights into model stability and expressive capacity through their representations.
- AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
- Demystifying Spectral Feature Learning for Instrumental Variable Regression
- Geometry-Aware Edge Pooling for Graph Neural Networks
- LoRA vs Full Fine-tuning: An Illusion of Equivalence
- Reparameterized LLM Training via Orthogonal Equivalence Transformation
- Spectral Analysis of Diffusion Models with Application to Schedule Design
- Spectral Analysis of Representational Similarity with Limited Neurons
- Spectral Conditioning of Attention Improves Transformer Performance