gradient variance
Gradient variance refers to variability in the computed gradients during optimization, which can impact convergence speed and stability in training algorithms. Managing this variance is key for efficient learning.
- ARIA: Training Language Agents with Intention-driven Reward Aggregation
- Gradient Variance Reveals Failure Modes in Flow-Based Generative Models
- MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
- Multi-Kernel Correlation-Attention Vision Transformer for Enhanced Contextual Understanding and Multi-Scale Integration
- Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference
- Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RL
- Second-Order Convergence in Private Stochastic Non-Convex Optimization
- Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations