model quality

3 papers

• Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo
• First Attentions Last: Better Exploiting First Attentions for Efficient Parallel Training
• The Leaderboard Illusion