What Is a Generative Model
Every model in the classical ML and deep learning sections so far predicts a label or value from an input. A generative model asks a different, harder question: can you produce a new example that looks like it came from the same distribution as the training data? Answering "yes" requires learning something classifiers never need — an actual model of what the data looks like.
Autoencoders
Train a network to copy its input to its output, and the task sounds trivially easy — the identity function does it perfectly. Force that copying to pass through a narrow bottleneck first, and suddenly the network has to decide what matters enough to keep. That forced compression is the entire idea behind autoencoders.
Variational Autoencoders
Autoencoders ended on an unsolved problem: the latent space has no known structure, so there's no principled way to pick a point to decode into a new sample. The variational autoencoder's fix is direct — force the latent space to match a known distribution during training, and sampling becomes as simple as drawing from that known distribution and decoding.
GANs
Every generative model so far has needed an explicit loss function measuring "how good is this sample." GANs replace that explicit loss with a second trained network, whose entire job is to learn what "real" looks like — and the generator improves purely by trying to fool it.
GAN Training Challenges
GANs are famously hard to train, and the failure modes are specific and recognisable. Worse: the standard debugging instinct — watch the loss curve — actively misleads here.
Normalizing Flows
Every generative family so far trades away exact likelihood for something else: GANs give up density entirely, VAEs settle for a lower bound. Normalizing flows refuse that trade — by restricting every layer to be invertible, they keep an exact, computable likelihood all the way through a deep, expressive transformation.
Diffusion Models
Generating an image from nothing in one shot is hard. Diffusion sidesteps the difficulty entirely: destroy an image with noise across many small steps, then train a network to undo just one of those small steps at a time — a thousand easy problems standing in for one hard one.
Sampling & Guidance
A trained diffusion model is only half the system. Diffusion Models trains a noise predictor — but how you turn that predictor into actual samples, how many steps you take, and what you condition on are all choices made after training, and they are where most of the practical control lives.
Flow Matching & Consistency Models
DDPM Sampling and Guidance got diffusion sampling down to tens of steps. This page covers the research direction aimed squarely at pushing that further — toward single-digit, and eventually single-step, generation, by rethinking what the network is trained to predict in the first place.
Multimodal Generation
How does a sentence become an image? Not through one monolithic model — a text-to-image system is three separately-motivated trained components, wired together, and most of what a system can and can't do is a direct consequence of how they're joined.
Evaluating Generative Models
A classifier has a right answer to check against. A generative model's output has no correct answer at all — "is this a good generated cat image" has no ground truth to compare against, only a distribution to compare against. Every metric in this page is a different proxy for that comparison, and every proxy can be gamed.