optimization dynamics
Optimization dynamics studies the behavior of optimization algorithms during the training of machine learning models, including how they converge, oscillate, or explore the search space, which is crucial for understanding and improving learning efficiency.
- Beyond Modality Collapse: Representation Blending for Multimodal Dataset Distillation
- Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel
- How Memory in Optimization Algorithms Implicitly Modifies the Loss
- Learning to Generalize: An Information Perspective on Neural Processes
- On Linear Mode Connectivity of Mixture-of-Experts Architectures
- On Linear Mode Connectivity of Mixture-of-Experts Architectures
- Some Optimizers are More Equal: Understanding the Role of Optimizers in Group Fairness