Embeddings: Meaning as Vectors
Words become points in space where similar meanings sit close together — and directions carry meaning, as in king − man + woman ≈ queen.
📄 Illustrated notes · every chapter as a picture · printable
Quick quiz
3 questions to check your understanding.
Go deeper
University-level written lectures in The AI Lecture Hall:
Transcript
Introduction. Neural networks work with numbers, but words, products and users are not numbers. Embeddings turn them into lists of numbers that capture meaning.
Meaning as position. Each word becomes a point in space. We draw two dimensions here, but real embeddings have hundreds. After training, words used in similar ways end up close together. Fruit near fruit, animals near animals, royalty near royalty. Nobody labelled these groups.
Analogies. Even directions can carry meaning. The step from man to woman points the same way as the step from king to queen. So king minus man plus woman lands close to queen. This famous example comes from word2vec.
Training. Embeddings start random. A model is trained on a task, such as predicting nearby words. Gradients nudge the vectors, and words that appear in similar contexts drift together. Meaning emerges from usage.
Uses. Embeddings power semantic search, recommendations and every language model. Multimodal models even place images and text in the same space, so a photo of a dog lands near the words a dog.
Recap. To recap. Embeddings map things to vectors. Similar meanings sit close together, directions can encode relationships, and all of it is learned from usage.