pruning
Pruning refers to the technique in machine learning, especially in neural networks, where less important weights or neurons are removed to simplify the model. This can lead to reduced computational costs and improved performance, particularly in deployment scenarios.
- A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
- Benford’s Curse: Tracing Digit Bias to Numerical Hallucination in LLMs
- Computation and Memory-Efficient Model Compression with Gradient Reweighting
- DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration
- DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs
- Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
- Efficient Low Rank Attention for Long-Context Inference in Large Language Models
- GradMetaNet: An Equivariant Architecture for Learning on Gradients
- Hyperbolic Dataset Distillation
- ModHiFi: Identifying High Fidelity predictive components for Model Modification
- Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
- Restoring Pruned Large Language Models via Lost Component Compensation