optimization algorithms
Procedures used to adjust the parameters of an AI model to minimize or maximize a certain objective function, key to effective training.
- A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias
- ASGO: Adaptive Structured Gradient Optimization
- Accelerated Distance-adaptive Methods for Hölder Smooth and Convex Optimization
- Accelerating Optimization via Differentiable Stopping Time
- Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
- Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
- Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions
- DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization
- Deployment Efficient Reward-Free Exploration with Linear Function Approximation
- Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity
- Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size
- Exploring Landscapes for Better Minima along Valleys
- FSEO: Few-Shot Evolutionary Optimization via Meta-Learning for Expensive Multi-Objective Optimization
- Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference
- Nested Learning: The Illusion of Deep Learning Architectures
- Neural Evolution Strategy for Black-box Pareto Set Learning
- New Perspectives on the Polyak Stepsize: Surrogate Functions and Negative Results
- Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis
- On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm
- Private Zeroth-Order Optimization with Public Data
- Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner
- Quasi-Self-Concordant Optimization with $\ell_{\infty}$ Lewis Weights
- Revisiting Consensus Error: A Fine-grained Analysis of Local SGD under Second-order Data Heterogeneity
- Robust and Diverse Multi-Agent Learning via Rational Policy Gradient
- Stochastic Gradients under Nuisances