optimization trajectory

3 papers

• Breaking the Frozen Subspace: Importance Sampling for Low-Rank Optimization in LLM Pretraining
• Exploring Tradeoffs through Mode Connectivity for Multi-Task Learning
• Stochastic Gradients under Nuisances