foundation model
/ fown-DAY-shun MOD-ul /
A foundation model is a single, very large model trained on a huge, broad pile of data — text, images, or both — so that it can later be adapted to many different tasks instead of being built from scratch for each one. The name captures the idea: it is a common foundation you build many specific applications on top of, the way one strong concrete base can support a house, a shop, or an office tower. The large language models behind today's chatbots are the best-known examples.
What makes it work is a two-stage recipe. First comes pretraining: the model digests an enormous, general corpus, learning rich patterns of language or vision without being aimed at any one job — this stage is hugely expensive and done once. Then comes adaptation: the same model is fine-tuned, prompted, or otherwise specialized for a particular task — summarizing legal documents, answering customer questions, captioning photos — at a tiny fraction of the cost. The 2018–2020 wave of such models, and the 2021 coinage of the term 'foundation model,' marked a shift from building one narrow model per task to reusing one general model everywhere.
The upside is reach and efficiency: one model can power countless downstream uses, often with surprising flexibility. The honest downsides are just as important. These models are expensive and energy-hungry to train; they absorb the biases and errors of their training data; they can state falsehoods fluently and confidently (so-called hallucination); and because so much rides on a few big models, their flaws propagate widely. They are powerful general-purpose tools — not oracles, and not minds.
A hospital, a law firm, and a game studio might all build on the same underlying language model. The hospital fine-tunes it on medical notes, the firm prompts it to draft contracts, the studio uses it to write dialogue. One costly foundation; three cheap, very different applications — and any flaw or bias in that foundation quietly shows up in all three.
One expensive base model, many cheap specialized uses — and shared flaws that travel with it.
A foundation model is 'general' in the sense of being reusable across tasks — not in the sense of artificial general intelligence. It is still a pattern-matcher that can be fluent and confidently wrong; broad usefulness is not understanding.