model depth
Refers to the number of layers in a neural network. Deeper models can capture more complex patterns but also require careful management of issues like overfitting and vanishing gradients.
- Counterfactual reasoning: an analysis of in-context emergence
- Exploring Diffusion Transformer Designs via Grafting
- Exploring Diffusion Transformer Designs via Grafting
- How to Scale Second-Order Optimization
- On the Robustness of Transformers against Context Hijacking for Linear Classification
- The Curse of Depth in Large Language Models
- What One Cannot, Two Can: Two-Layer Transformers Provably Represent Induction Heads on Any-Order Markov Chains