gradient descent
Gradient descent is an optimization algorithm used to minimize the loss function of machine learning models by iteratively adjusting the model parameters in the direction of the steepest descent, based on the computed gradients.
- A Closer Look at NTK Alignment: Linking Phase Transitions in Deep Image Regression
- A Minimalist Example of Edge-of-Stability and Progressive Sharpening
- Abstain Mask Retain Core: Time Series Prediction by Adaptive Masking Loss with Representation Consistency
- Any-stepsize Gradient Descent for Separable Data under Fenchel–Young Losses
- Asymptotically Stable Quaternion-valued Hopfield-structured Neural Network with Periodic Projection-based Supervised Learning Rules
- Attention with Trained Embeddings Provably Selects Important Tokens
- Auto-Compressing Networks
- Auto-Compressing Networks
- BIPNN: Learning to Solve Binary Integer Programming via Hypergraph Neural Networks
- Bayes optimal learning of attention-indexed models
- Benign Overfitting in Single-Head Attention
- Causal Discovery and Inference through Next-Token Prediction
- Complexity Scaling Laws for Neural Models using Combinatorial Optimization
- Composing Global Solutions to Reasoning Tasks via Algebraic Objects in Neural Nets
- Convergence Rates for Gradient Descent on the Edge of Stability for Overparametrised Least Squares
- Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study
- Efficient Data Selection at Scale via Influence Distillation
- Fast exact recovery of noisy matrix from few entries: the infinity norm approach
- Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers
- Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules
- Guarantees for Alternating Least Squares in Overparameterized Tensor Decompositions
- Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time
- How Does Label Noise Gradient Descent Improve Generalization in the Low SNR Regime?
- How Memory in Optimization Algorithms Implicitly Modifies the Loss
- Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression
- Learning Provably Improves the Convergence of Gradient Descent
- Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions
- Let a Neural Network be Your Invariant
- Local Curvature Descent: Squeezing More Curvature out of Standard and Polyak Gradient Descent
- MAP Estimation with Denoisers: Convergence Rates and Guarantees
- Multiclass Loss Geometry Matters for Generalization of Gradient Descent in Separable Classification
- New Perspectives on the Polyak Stepsize: Surrogate Functions and Negative Results
- Non-Convex Tensor Recovery from Tube-Wise Sensing
- Non-Singularity of the Gradient Descent Map for Neural Networks with Piecewise Analytic Activations
- On the Edge of Memorization in Diffusion Models
- Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification
- Quantitative convergence of trained neural networks to Gaussian processes
- Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions
- Robust LLM Alignment via Distributionally Robust Direct Preference Optimization
- Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf’s Law
- Temperature is All You Need for Generalization in Langevin Dynamics and other Markov Processes
- The Computational Advantage of Depth in Learning High-Dimensional Hierarchical Targets
- The Implicit Bias of Structured State Space Models Can Be Poisoned With Clean Labels
- Towards Understanding Transformers in Learning Random Walks
- Towards the Resistance of Neural Network Fingerprinting to Fine-tuning
- Trained Mamba Emulates Online Gradient Descent in In-Context Linear Regression
- Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization
- Transformers are almost optimal metalearners for linear classification
- Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability
- Variational Inference with Mixtures of Isotropic Gaussians
- Variational Learning Finds Flatter Solutions at the Edge of Stability