from autocomplete to LLM

The autocomplete on your phone guesses the next word from the last few you typed, suggesting you after thank. An LLM is the same idea taken to an extreme. Instead of looking at three words, it can look at thousands; instead of a tiny table of common phrases, it has billions of parameters capturing patterns across the whole internet's worth of text.

Scale changes the character of the task. Phone autocomplete only ever finishes a phrase. But when a model can condition on pages of context and has absorbed enough text, finishing the sentence well sometimes requires answering a question, following an instruction, or writing working code. The mechanism stayed predict the next word; the competence grew until it stopped looking like autocomplete.