adam

5 papers

• A geometric framework for momentum-based optimizers for low-rank training
• Flat Channels to Infinity in Neural Loss Landscapes
• In Search of Adam’s Secret Sauce
• In Search of Adam’s Secret Sauce
• Understanding Adam Requires Better Rotation Dependent Assumptions