PCA and SVD
Principal component analysis answers a simple question with surprisingly deep machinery: which few directions in a high-dimensional dataset capture most of what's actually going on? The answer — rotate onto the axes of greatest variance — turns out to be one of the most reused mathematical results in all of machine learning.