model design
- Cognitive Mirrors: Exploring the Diverse Functional Roles of Attention Heads in LLM Reasoning
- DBLoss: Decomposition-based Loss Function for Time Series Forecasting
- From Black-box to Causal-box: Towards Building More Interpretable Models
- Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search
- Pool Me Wisely: On the Effect of Pooling in Transformer-Based Models
- The emergence of sparse attention: impact of data distribution and benefits of repetition
- The emergence of sparse attention: impact of data distribution and benefits of repetition