condition number
The condition number is a measure of how sensitive a function or model is to changes in input, indicating potential numerical stability issues in optimization processes.
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks
- Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
- Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks
- Fast exact recovery of noisy matrix from few entries: the infinity norm approach
- Finding Low-Rank Matrix Weights in DNNs via Riemannian Optimization: RAdaGrad and RAdamW
- Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression
- Learning Sparse Approximate Inverse Preconditioners for Conjugate Gradient Solvers on GPUs
- On the Convergence of Stochastic Smoothed Multi-Level Compositional Gradient Descent Ascent
- Spectral Conditioning of Attention Improves Transformer Performance
- Stepsize anything: A unified learning rate schedule for budgeted-iteration training