Research paper

Efficient Estimation of Word Representations in Vector Space

Mikolov, Chen, Corrado, Dean — Google

arXiv 1301.3781 · 2013

1 / 9
Why this matters
🔢
→
🗺️

Words as numbers can't capture meaning

Computers need a way to know "king" relates to "queen"

2 / 9
The gap
Before cat=3 dog=7 just IDs

Each word is a random ID — no similarity

→
After near = similar

Similar words sit close in vector space

3 / 9
Core idea

Learn word meaning from nearby words in text

Like guessing "bank" means river or money from context

4 / 9
How it works
Huge text corpus
→
Sliding window
→
Predict neighbors
→
Word vectors

Train fast on billions of words — no heavy neural nets needed

5 / 9
Key mechanism
Skip-gram king queen royal throne center → predict context CBOW the cat on mat context → predict center

Two lightweight architectures — both beat older neural language models

6 / 9
Results
1.6B
words trained on
<1 day
training time
~300
vector dimensions
NNLM
CBOW
Skip-gram
7 / 9
So what?
⚡

Fast enough for web-scale data

🧠

Embeddings power modern NLP

➕

king − man + woman ≈ queen

8 / 9
Remember this

You are what you're surrounded by — for words too.

9 / 9

← → or click to navigate