Language Modeling Basics
Every generative model in this section — from a bigram model to GPT — is doing the same thing: predicting the next token given everything before it. That single objective, applied one token at a time, is the entire engine behind modern text generation; everything else in this section is about how to do that prediction better.