large language model
/ larj LANG-gwij MOD-uhl /
A large language model is an enormous web of numbers trained on a simple game: read a stretch of text and guess the next word. Play that game over and over across a sizable slice of everything humans have written — books, code, conversations, the open web — and the model slowly learns the patterns of language. It's a bit like the autocomplete on your phone, but grown to staggering size, so that instead of finishing a text it can write an essay, translate a poem, or explain a recipe.
Under the hood sits an architecture called a transformer, whose key trick is attention: when reading, the model weighs which earlier words matter most for guessing what comes next, letting it keep track of meaning across long passages. With billions of adjustable knobs and oceans of training text, it builds a rich internal map of how words relate.
Here's the surprise that makes these models matter: nobody programmed grammar, facts, or reasoning into them by hand. Those abilities simply emerge once the model and its training are large enough — a jump in scale brings a jump in skill. A common misconception is that the model looks things up or truly understands; really it predicts plausible words. That's why it can sound confident yet be wrong — a slip people call hallucination.
Given "The cat sat on the ___", the model assigns high probability to "mat", lower to "roof", and tiny probability to "democracy" — then picks from that distribution.
Prediction, not recall: an LLM models the odds of the next token.
The breakthrough traces to a 2017 paper, "Attention Is All You Need," which introduced the transformer; the "large" in the name refers to sheer scale — the parameter count and training data. "GPT," as in OpenAI's models, stands for Generative Pre-trained Transformer and names just one well-known family of large language models — many leading ones, such as Gemini, Claude, and LLaMA, are not GPTs.