Deep LearningBeginner1:24 video6 chapters

Embeddings: Meaning as Vectors — lecture notes

Words become points in space where similar meanings sit close together — and directions carry meaning, as in king − man + woman ≈ queen.

▶ Watch the animated lecture

0:001. Introduction

Introduction — Embeddings: Meaning as Vectors

Neural networks work with numbers, but words, products and users are not numbers. Embeddings turn them into lists of numbers that capture meaning.

0:102. Meaning as position

Meaning as position — Embeddings: Meaning as Vectors

Each word becomes a point in space. We draw two dimensions here, but real embeddings have hundreds. After training, words used in similar ways end up close together. Fruit near fruit, animals near animals, royalty near royalty. Nobody labelled these groups.

0:283. Analogies

Analogies — Embeddings: Meaning as Vectors

Even directions can carry meaning. The step from man to woman points the same way as the step from king to queen. So king minus man plus woman lands close to queen. This famous example comes from word2vec.

0:444. Training

Training — Embeddings: Meaning as Vectors

Embeddings start random. A model is trained on a task, such as predicting nearby words. Gradients nudge the vectors, and words that appear in similar contexts drift together. Meaning emerges from usage.

0:585. Uses

Embeddings power semantic search, recommendations and every language model. Multimodal models even place images and text in the same space, so a photo of a dog lands near the words a dog.

1:136. Recap

To recap. Embeddings map things to vectors. Similar meanings sit close together, directions can encode relationships, and all of it is learned from usage.

Key takeaways

Embeddings represent items as vectors in which similar meanings are close.
Directions can encode relationships: king − man + woman ≈ queen.
They are learned by training on tasks such as predicting context.
They power search, recommendations and language models — and can inherit bias from data.

Check yourself

In a good embedding space, “mango” should be close to…
Show answer
“apple” — Words used in similar contexts end up nearby.
king − man + woman is closest to…
Show answer
queen — The gender direction is shared between the pairs.
How are embedding vectors usually obtained?
Show answer
Learned during training — They start random and are adjusted by gradient descent.

Go deeper

Embeddings: Turning Discrete Things into Meaningful Vectors · The AI Lecture Hall
Word2Vec: Learning Word Embeddings from Context · The AI Lecture Hall
Information Retrieval and Semantic Search · The AI Lecture Hall