Limitations, Hallucination & Risks

hallucination (basics)

Hallucination is when a language model states something false as if it were plainly true. Ask it for the population of a small town, the plot of a book it half-remembers, or a quote from a person, and it may hand you a clean, confident, completely invented answer. Nothing about the reply looks wrong — there is no stutter, no hedge, no flashing warning. That smoothness is exactly what makes it dangerous.

The key thing to absorb is that this is not a bug that a patch will remove. The model was built to produce text that sounds right, not text that is verified against the world. When it does not know, it does not stay silent — it generates the most plausible-looking continuation, and a plausible-looking falsehood is still a falsehood. Treat every factual claim from an LLM as a confident draft that still needs checking.

The word hallucination is a metaphor — the model is not perceiving anything. It is simply filling a gap with its best statistical guess.

Also called
confabulation