student model
In a teacher-student framework for machine learning, the student model refers to the simpler model being trained to replicate the behavior of a more complex 'teacher' model. This approach is often used in knowledge distillation to create efficient models without sacrificing performance.
- Cross-Modal Representational Knowledge Distillation for Enhanced Spike-informed LFP Modeling
- Hawaii: Hierarchical Visual Knowledge Transfer for Efficient Vision-Language Models
- KINDLE: Knowledge-Guided Distillation for Prior-Free Gene Regulatory Network Inference
- On the Mechanisms of Weak-to-Strong Generalization: A Theoretical Perspective
- SDPGO: Efficient Self-Distillation Training Meets Proximal Gradient Optimization
- Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation