bag-of-words
/ BAG-uv-WURDZ /
The bag-of-words model is the simplest way to turn a document into numbers: dump all its words into a bag, shake it up, and just count how many times each word appears — throwing away the order entirely. "The dog bit the man" and "The man bit the dog" become the exact same bag: {the: 2, dog: 1, bit: 1, man: 1}. To the model these two sentences are identical, which is both the method's great convenience and its glaring blind spot.
Concretely, you fix a vocabulary — every word the model knows — and represent each document as a long list of counts, one slot per vocabulary word, mostly zeros. That list is a vector a machine can compute with: compare documents, feed a classifier, search for matches. Despite ignoring grammar, word order, and meaning, bag-of-words is shockingly effective for tasks where the mere presence of words is informative, like sorting email into spam or topic-tagging news articles.
It is worth knowing because it is the honest baseline of text processing — the thing you try first, the yardstick fancier methods must beat. Its weaknesses are equally instructive: it cannot tell "not good" from "good," it treats "happy" and "joyful" as unrelated strangers, and the vectors balloon to enormous, mostly-empty sizes. Weighting schemes like TF-IDF soften the first problem; embeddings address the second.
Vocabulary [I, love, hate, cats, dogs]. "I love cats" → [1,1,0,1,0]. "I hate cats I hate dogs" → [2,0,2,1,1]. Two documents, now just two lists of counts a classifier can chew on.
Each document becomes a vector of word counts — order and grammar discarded.
Remember its fatal blind spot: bag-of-words throws away word order, so "not good" and "good" look almost the same to it. It is a strong, cheap baseline — not a model of meaning.