Generalization

concepts · 4 notes linked

Related: Deep Learning · Google · Arxiv · Memorization · Deepmind · Regularization · Bias Variance Tradeoff · Double Descent

Notes

• A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning — Survey of overparameterized ML theory explaining double descent and interpolation generalization
• A New Study by Google and DeepMind Introduces Geometric Complexity (GC) for Neural Network Analysis and Understanding of Deep Learning Models — Geometric Complexity measure explains regularization and double-descent in neural networks
• Deep Learning Through the Lens of Example Difficulty — Prediction depth as per-example measure of deep learning difficulty
• Keras inventor Chollet charts a new direction for AI: a Q&A — Chollet critiques deep learning limits and proposes ARC generalization benchmark