Shiwei Liu

3 papers University of Oxford

• AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs
• GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
• The Curse of Depth in Large Language Models