memory cost

3 papers

• Breaking the Frozen Subspace: Importance Sampling for Low-Rank Optimization in LLM Pretraining
• GraSS: Scalable Data Attribution with Gradient Sparsification and Sparse Projection
• Transformers Learn Faster with Semantic Focus