gradient flow
The path and magnitude of gradients during optimization, particularly in deep learning. Proper gradient flow is vital for effective weight updates and avoiding issues like vanishing or exploding gradients during training.
- A Theoretical Framework for Grokking: Interpolation followed by Riemannian Norm Minimisation
- Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies
- Attention with Trained Embeddings Provably Selects Important Tokens
- Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models
- Convergence of the Gradient Flow for Shallow ReLU Networks on Weakly Interacting Data
- Delving into Cascaded Instability: A Lipschitz Continuity View on Image Restoration and Object Detection Synergy
- Flat Channels to Infinity in Neural Loss Landscapes
- From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics
- From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics
- Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel
- HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization
- Impact of Layer Norm on Memorization and Generalization in Transformers
- Leveraging Depth and Language for Open-Vocabulary Domain-Generalized Semantic Segmentation
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data