Normalization Layers
Networks deeper than twenty or so layers were, for years, essentially untrainable — every layer's input distribution kept shifting as the layers below it updated, forcing each layer to constantly re-adapt to a moving target. Batch normalisation, and the family of normalisation layers it started, fixed this directly by re-standardising activations at every layer, which is what let networks scale to hundreds of layers.