GloVe
/ GLUHV /
GloVe is a close cousin of word2vec: another way to turn each word into a vector of numbers so that similar words sit near one another. The two share the same goal but reach it from different angles. Where word2vec slides a small window across the text and learns from local guessing games, GloVe steps back and looks at the whole corpus at once, building a giant table of how often each pair of words shows up together — the co-occurrence counts — and then fitting vectors to explain those counts.
The name says it: Global Vectors. The intuition is that the ratio of co-occurrences carries meaning. "Ice" appears with "solid" much more than "steam" does; "steam" appears with "gas" much more than "ice" does; and both appear with "water" about equally. GloVe is designed so that the relationships between word vectors line up with exactly these ratios. The result, like word2vec, supports the famous analogy arithmetic and clusters related words together.
Introduced by Stanford researchers in 2014, GloVe was for years a go-to set of pretrained vectors you could download and drop into a project. In practice it and word2vec perform comparably; which wins depends on the data and the task, not on grand principle. And GloVe shares the same honest limits: one vector per word (so word senses blur), and any bias in the source text gets baked into the geometry.
Consider how often each word co-occurs with "water": ice does, steam does, about equally. But ice co-occurs far more with "solid" and steam far more with "gas." GloVe fits vectors so those telling ratios become geometric distances.
GloVe learns from co-occurrence ratios across the whole corpus at once.
GloVe vs word2vec is not a battle of right and wrong — they encode the same kind of meaning by different routes and usually perform about the same. Both predate context-aware embeddings and give each word a single, sense-blind vector.