Bias-Variance Tradeoff
A model can be wrong in exactly two ways: it can be too simple to capture the real pattern (bias), or too sensitive to the particular training sample it happened to see (variance). Diagnosing which one you're facing tells you whether to add capacity or add data — the two most common fixes are not interchangeable.
Overfitting and Regularization
A model that memorises its training set — including its noise and its idiosyncrasies — is worthless the moment it sees a new example. Regularisation is the collection of techniques that stop a model from doing that, by encoding a preference for simpler explanations somewhere in the loss, the data, or the training procedure itself.