in-context learning

In-context learning is the striking ability of a large model to pick up a new task from a few examples shown in the prompt itself — without any training, without changing a single one of its internal numbers. You show it three examples of translating slang into formal English, and on the fourth it just does it. It looks like the model learned something in those few seconds, but nothing inside it actually changed; it inferred the pattern from the examples sitting in front of it and continued accordingly.

The mechanism is best thought of as very flexible pattern-completion. During pretraining the model saw countless passages that contained examples followed by more of the same kind, so when you lay out "input, output, input, output, input —", finishing the pattern is exactly the kind of continuation it became excellent at. "Zero-shot" means giving only the instruction; "few-shot" means giving a handful of worked examples first. This is what lets one frozen model handle thousands of different tasks just by changing the wording you feed it.

Two honest points. First, because nothing is stored, the "learning" vanishes the moment the prompt ends — the next conversation starts blank. Second, in-context learning is real but fragile: it is sensitive to the order of your examples, their format, even small wording changes, and it can latch onto a superficial pattern rather than the one you intended. It is a wonderful convenience, not a substitute for genuine training when you need reliability.