Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon
convergence ratescurse of dimensionalityedge-of-stabilityflatnessgeneralization gapimplicit biasloss curvaturemean-squared errorminima stabilityminimax lower boundmultivariate inputsneural shatteringnonparametric function estimationoverparameterized networksupper and lower bounds
We study the implicit bias of flatness / low (loss) curvature and its effects on generalization in two-layer overparameterized ReLU networks with multivariate inputs