momentum

5 papers

• Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size
• Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules
• How Memory in Optimization Algorithms Implicitly Modifies the Loss
• Streaming Federated Learning with Markovian Data
• Understanding outer learning rates in Local SGD