gradient descent dynamics

3 papers

• From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning
• Multi-head Transformers Provably Learn Symbolic Multi-step Reasoning via Gradient Descent
• Understanding Softmax Attention Layers:\\ Exact Mean-Field Analysis on a Toy Problem