A made-up word for a real problem
Ask a chatbot for the population of a small town, or for a paper that supports your argument, and sometimes it answers smoothly and completely — with a number or a title that simply does not exist. This is what people mean by hallucination: the model generates fluent, plausible text that is not grounded in any real fact. It is not lying, because lying requires knowing the truth and choosing to hide it. The model has no such inner state. It is producing the most likely-sounding continuation, and 'most likely-sounding' is not the same as 'true'.
Why it is built into the machine
To see why language models hallucinate, remember what they actually do: predict the next token over and over. During training the model learns the shape of human text — its grammar, its rhythms, the way citations and statistics tend to appear. It is rewarded for sounding right, never directly for being right. So when it has not memorized a fact, it does not stop and say 'I don't know'. It fills the gap with text that has the right shape: a real-looking author name, a believable year, a clean round number.
Diagram of an autoregressive loop: the model outputs a token, which is fed back as input for the next prediction.
This is the kernel of truth behind the dismissive nickname stochastic parrot: a model that stitches together patterns of language without an underlying model of the world will, by construction, sometimes stitch together something false. Hallucination is not a rare bug bolted on by accident; it is the same generative talent that lets the model write a poem, pointed at a question where it has no anchor.
All the model is trained to maximize is the probability of the next token given the previous ones — nowhere does it check whether the result is true.
Not all hallucinations are the same
A useful first cut at the types of hallucination separates two questions. First: is the output unfaithful to its source (it contradicts the document you gave it) or unfaithful to the world (it contradicts reality)? Second: did the model contradict something it was told, or invent something out of thin air? You will hear these called faithfulness versus factuality, and intrinsic versus extrinsic.
- Intrinsic: the answer contradicts the very text it was given (you paste an article saying 2019 and it summarizes it as 2021).
- Extrinsic: the answer adds claims that the source neither states nor implies (a plausible extra 'fact' that came from nowhere).
- Factual error: the claim is simply false about the world, regardless of any source.
- Faithfulness error: the claim may even be true in general, but it is not supported by the material the model was supposed to use.
The dangerous part: it sounds sure
What makes hallucination hazardous is not just that the model is wrong, but that it is wrong with a straight face. This is confident wrongness: the same calm, authoritative tone wraps a correct answer and a fabricated one. Humans use hesitation, hedging, and tone as cues for how much to trust a claim — and a fluent model strips those cues away.
Underneath, the model does carry a rough sense of its own uncertainty in its token probabilities, but a well-tuned chat model is often poorly calibrated for honesty: it has been trained to be helpful and assertive, so its stated confidence and its actual reliability can drift apart. Closing that gap — getting stated confidence to track real uncertainty — is an open research problem, which is exactly why you cannot use the model's tone as a truth meter.
A model's 'confidence' is just its largest token probability; a poorly calibrated chat model can keep that high even when the answer is wrong.
What the rest of this track covers
Hallucination is the headline failure, but it is one of a family. The next guide breaks down the specific ways facts go wrong. After that we look beyond facts — at bias, toxic output, and faulty reasoning — then at the brittle edges where prompts and context windows fail, and finally at the wider risks to privacy, copyright, the environment, and the information ecosystem. The thread running through all of it: these systems are remarkably useful and genuinely fallible, and using them well means holding both truths at once.