stochastic gradient descent
An optimization algorithm that updates model parameters using random subsets of data, widely used for training large-scale neural networks due to its efficiency.
- A Unified Analysis of Stochastic Gradient Descent with Arbitrary Data Permutations and Beyond
- A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias
- Adaptive Sigmoid Clipping for Balancing the Direction–Magnitude Mismatch Trade-off in Differentially Private Learning
- Asymptotic theory of SGD with a general learning-rate
- Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks
- Continuous-time Riemannian SGD and SVRG Flows on Wasserstein Probabilistic Space
- Contribution of task-irrelevant stimuli to drift of neural representations
- Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions
- Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacy
- Decreasing Entropic Regularization Averaged Gradient for Semi-Discrete Optimal Transport
- Differentiable Sparsity via $D$-Gating: Simple and Versatile Structured Penalization
- Dimension-adapted Momentum Outscales SGD
- Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime
- FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA
- Functional Scaling Laws in Kernel Regression: Loss Dynamics and Learning Rate Schedules
- Gaussian Approximation and Concentration of Constant Learning-Rate Stochastic Gradient Descent
- GeoClip: Geometry-Aware Clipping for Differentially Private SGD
- Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm
- Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning
- Optimal Rates in Continual Linear Regression via Increasing Regularization
- PaZO: Preconditioned Accelerated Zeroth-Order Optimization for Fine-Tuning LLMs
- Private Statistical Estimation via Truncation
- Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes
- Statistical Guarantees for High-Dimensional Stochastic Gradient Descent
- Stochastic Optimization in Semi-Discrete Optimal Transport: Convergence Analysis and Minimax Rate
- The Rich and the Simple: On the Implicit Bias of Adam and SGD
- Tight High-Probability Bounds for Nonconvex Heavy-Tailed Scenario under Weaker Assumptions
- Time-uniform and Asymptotic Confidence Sequence of Quantile under Local Differential Privacy
- Uncertainty Quantification with the Empirical Neural Tangent Kernel
- Understanding Adam Requires Better Rotation Dependent Assumptions
- When Does Curriculum Learning Help? A Theoretical Perspective