Model Capacity and Scaling
How big should the model be? The classical answer from Bias-Variance Tradeoff — bigger risks overfitting past some point — turns out to be incomplete for the over-parameterised networks that now dominate deep learning, where a second, deeper descent in test error can appear past the point where the classical U-curve says things should be getting worse.