overfitting
Overfitting is what happens when a learning model studies its practice examples so obsessively that it memorizes them instead of understanding them — quirks, noise, lucky accidents and all. Picture a student who crams a past exam paper word for word: ask the exact questions and they ace it, but change a single number and they're lost. The model aced the practice and flunked the real thing.
This matters because the whole point of machine learning is to do well on data it has never seen — to generalize. An overfit model looks brilliant on its training data and disappointing everywhere else, so its glowing practice scores are a mirage. The fix is to stop the model from clinging too tightly: feed it more and more varied examples, keep it simpler, or apply regularization — a gentle penalty that discourages it from chasing every last wrinkle in the data.
A common mix-up: overfitting is not the same as a model simply being wrong. An overfit model is, if anything, too right about the training set — it has learned the answers by heart. Its twin sin is underfitting, where the model is too crude to capture even the real pattern. Good learning threads the needle between the two.
A spam filter trained only on emails containing the word "Nigeria" flags every message from a colleague on holiday there — it memorized a clue instead of learning what spam is.
Memorizing a coincidence in the training data instead of the real pattern.
The term comes from statistics, where a curve that bends to pass through every single data point "fits" the sample too well — it traces the noise rather than the signal. In modern deep learning the everyday safeguards are regularization, dropout, and an early stop in training, all watched over by a held-out validation set the model is never allowed to memorize.