information preservation

3 papers

• Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
• GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling
• H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition