n-gram

An n-gram is just a short, sliding window of n items in a row — usually n words, sometimes n letters. Slide a window two-words-wide across "the cat sat on the mat" and you get the bigrams (2-grams): "the cat," "cat sat," "sat on," "on the," "the mat." One word is a unigram, two a bigram, three a trigram, and so on. The point of grabbing chunks instead of single words is that order starts to matter: "not good" survives as a unit, where a plain bag of words would lose it.

N-grams power a beautifully simple idea about language: you can guess the next word by counting what usually followed in the past. Having seen huge amounts of text, a program notices that "a cup of" is very often followed by "coffee" or "tea," rarely by "democracy." Count enough n-grams and you have a crude but real model of how language flows — this is exactly how early autocomplete, spelling correction, and speech recognition worked.

The catch is a brutal trade-off. Bigger n captures more context but explodes in count: the number of possible n-grams grows astronomically, and almost all of them never appear in your data, so you are forever short of examples (a problem called sparsity). That ceiling — n-grams can only ever "see" a few words back — is precisely what neural language models were invented to break, by learning smooth representations instead of brittle counts.