distillation
A process in model compression where a smaller model (the student) is trained to replicate the performance of a larger, pre-trained model (the teacher). This approach aims to retain the teacher's knowledge while reducing the model's size.
- AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning
- AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
- ChemOrch: Empowering LLMs with Chemical Intelligence via Groundbreaking Synthetic Instructions
- Distillation Robustifies Unlearning
- Distilling LLM Agent into Small Models with Retrieval and Code Tools
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
- Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
- Learning to Focus: Causal Attention Distillation via Gradient‐Guided Token Pruning
- Learning to Integrate Diffusion ODEs by Averaging the Derivatives
- LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervision
- Mean Flows for One-step Generative Modeling
- On the creation of narrow AI: hierarchy and nonlocality of neural network skills
- One-Step Offline Distillation of Diffusion-based Models via Koopman Modeling
- PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding
- QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation
- Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generation
- SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent Trajectories
- The Best Instruction-Tuning Data are Those That Fit
- Universal Cross-Tokenizer Distillation via Approximate Likelihood Matching