Glossary

Natural Language Processing — getting computers to work with human language.

The field this paper lives in; word vectors are one of its building blocks.

A list of numbers that represents a word. Similar words get similar lists.

Like GPS coordinates for meaning — close coordinates = related words.

Meaning spread across many numbers, not stored in a single slot.

A portrait made of thousands of pixels — no one pixel is "the nose."

Predicts the next word by counting how often word sequences appear in text.

Autocomplete based purely on statistics — no understanding of meaning.

Neural Network Language Model — learns word vectors while predicting the next word.

A student who learns vocabulary only as a side effect of finishing sentences.

Continuous Bag-of-Words — uses surrounding words to predict the center word.

Fill-in-the-blank: "The cat ___ on the" → you guess "sat."

Uses one center word to predict each surrounding word separately.

Given "king," guess what words typically appear nearby.

How many neighboring words on each side count as "context" (typically 5–10).

How far you peek left and right when reading one word in a sentence.

A large body of text used for training — e.g. Google News (6 billion tokens).

The textbook the model reads cover to cover, many times.

The set of words the model knows — here, the 1 million most frequent words.

The dictionary the model keeps — rare words may be dropped.

How many numbers in each word vector — typically 300–1000 in this paper.

More dimensions = finer detail, but more compute and data needed.

Turns scores into probabilities over all words in the vocabulary.

A weighted vote across every word — highest probability wins.

Learning by repeatedly adjusting vectors after each small batch of examples.

Fix mistakes one at a time instead of waiting for the whole book.

Meaning-based relationships — e.g. Athens:Greece :: Oslo:Norway.

Do these words mean related things in the real world?

Grammar-based relationships — e.g. walking:walked :: swimming:swam.

Do these words follow the same grammatical pattern?

Add/subtract word vectors to find relationships: king − man + woman ≈ queen.

Directions in vector space encode concepts like "gender" or "capital-of."

A model that scores word pairs using dot products — no hidden neural layer.

Matchmaking by compatibility score instead of a deep interview.

Randomly skip very frequent words ("the", "a") during training to speed up learning.

Don't re-read "the" a billion times — focus on informative words.

Google's distributed training framework — many machines train the model in parallel.

Many cooks working on the same soup simultaneously.