word2vec
/ WURD-too-VEK /
Word2vec is a famous trick for teaching a computer the meaning of words by watching the company they keep. The old idea behind it: "you shall know a word by the company it keeps." If "coffee" and "tea" both tend to appear near "cup," "hot," and "morning," then they probably mean something similar — and word2vec turns that hunch into numbers. Each word becomes a list of numbers (a vector, often a few hundred of them), placed so that words used in similar ways end up near each other in a vast invisible space.
Mechanically, word2vec trains a small neural network on a simple guessing game over plain text: given a word, predict its neighbors (the skip-gram version), or given the neighbors, predict the missing word (the CBOW version). Nobody labels anything — the text itself supplies the answers. After chewing through billions of words, the network's internal settings, read off as vectors, capture surprisingly rich meaning. The celebrated party trick is that arithmetic on these vectors works: king − man + woman lands very close to queen.
Released by Google researchers in 2013, word2vec was a turning point: it made dense, reusable word meanings cheap to compute and easy to plug into other systems. Two honest caveats, though. It gives each word exactly one vector, so "bank" (river) and "bank" (money) get blurred into one muddle — newer models fix this by reading context. And because it learns from human text, it also soaks up human prejudices, happily reproducing biases hidden in the data.
Take the vectors for "king," "man," and "woman." Compute king − man + woman. The closest word vector to the result is "queen" — the geometry has captured a gender relationship without anyone ever explaining it.
"king − man + woman ≈ queen" — meaning showing up as arithmetic.
Word2vec gives each word one fixed vector regardless of context, so it cannot tell the two senses of "bank" apart. Context-sensitive models (like BERT) later solved this — but word2vec's core idea, meaning as geometry, lives on in every embedding today.