Training a model is repeated use of one operation: measure the slope of the loss with respect to each parameter, then step in the direction that makes the loss smaller. Every optimiser in this knowledge base — from plain gradient descent to Adam — is a variation on that one move. This page derives the machinery once so later pages can use it without re-deriving it.
For two decades, networks deeper than a handful of layers simply refused to train — not because the architecture was wrong, but because the gradient signal reaching the earliest layers had either shrunk to numerical zero or grown to numerical infinity by the time it arrived there. Understanding exactly why this happens is what makes the fixes (initialisation, activations, normalisation, residual connections) make sense as one coherent story rather than a grab-bag of tricks.