Learning Rate Schedules
The learning rate is the single hyperparameter that most determines whether training works at all, and holding it fixed for the entire run is rarely the best choice. Every schedule on this page is a variation of the same idea: large steps early, when you're far from a good solution and want to explore quickly, small steps late, when you're close and want to settle precisely.