Generative vs Discriminative Models
One kind of model learns where the boundary is; the other learns what the data looks like — and can create new examples.
📄 Illustrated notes · every chapter as a picture · printable
Quick quiz
3 questions to check your understanding.
Go deeper
University-level written lectures in The AI Lecture Hall:
Transcript
Introduction. Generative AI can write, draw and compose. What makes a model generative? It comes down to what the model learns about the data.
Two ways to learn. A discriminative model only learns the boundary between classes: is this pink or blue? A generative model learns what each class looks like, here as a cloud of probability. Because it knows the shape of the data, it can draw brand new samples from it, the amber circles.
The difference. Discriminative models learn the probability of a label given the data. Generative models learn the probability of the data itself. That is harder, but it lets them create new images, text and sound.
Families. Generative AI grew through several families: variational autoencoders in 2013, GANs in 2014, generative pre-trained transformers from 2018, diffusion models from 2020, and in 2022 Stable Diffusion and ChatGPT brought them to everyone.
Recap. To recap. Discriminative models learn boundaries. Generative models learn the data itself, which lets them sample new examples. VAEs, GANs, diffusion models and language models are all generative.