heavy-tailed distributions
Statistical distributions characterized by a large number of occurrences of extreme values, which can impact model training and evaluation in machine learning.
- AI Progress Should Be Measured by Capability-Per-Resource, Not Scale Alone: A Framework for Gradient-Guided Resource Allocation in LLMs
- Exploring and Exploiting Model Uncertainty in Bayesian Optimization
- Put CASH on Bandits: A Max K-Armed Problem for Automated Machine Learning
- Sampling by averaging: A multiscale approach to score estimation
- Slow Transition to Low-Dimensional Chaos in Heavy-Tailed Recurrent Neural Networks
- The Quotient Bayesian Learning Rule