learning dynamics
Learning dynamics refer to the changes and processes that occur as an AI model learns over time, including the adjustments in weights, convergence of the algorithm, and the impact of data distribution. Understanding learning dynamics helps researchers optimize training strategies and model performance.
- A Closer Look at NTK Alignment: Linking Phase Transitions in Deep Image Regression
- A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models
- Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation
- Curl Descent : Non-Gradient Learning Dynamics with Sign-Diverse Plasticity
- Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments
- Differentiable Sparsity via $D$-Gating: Simple and Versatile Structured Penalization
- Dimensional Collapse in VQVAEs: Evidence and Remedies
- Evolution of Information in Interactive Decision Making: A Case Study for Multi-Armed Bandits
- Flexible inference for animal learning rules using neural networks
- From Average-Iterate to Last-Iterate Convergence in Games: A Reduction and Its Applications
- Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning
- Learning Dynamics of RNNs in Closed-Loop Environments
- Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning
- Non-Singularity of the Gradient Descent Map for Neural Networks with Piecewise Analytic Activations
- Optimal Dynamic Regret by Transformers for Non-Stationary Reinforcement Learning
- Prior Forgetting and In-Context Overfitting
- Spike-timing-dependent Hebbian learning as noisy gradient descent
- Tapered Off-Policy REINFORCE - Stable and efficient reinforcement learning for large language models
- The Computational Advantage of Depth in Learning High-Dimensional Hierarchical Targets
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness
- The emergence of sparse attention: impact of data distribution and benefits of repetition
- The emergence of sparse attention: impact of data distribution and benefits of repetition
- Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding
- What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov Chains
- When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective