Overfitting

Overfitting is when a model learns the training data too well, memorizing its random quirks and noise instead of the real underlying pattern. The giveaway is a model that scores beautifully on the data it was trained on but does badly on fresh data it has never seen.

It is the central danger of flexible models with many parameters and little data — like a student who memorizes the practice exam answers but cannot solve new problems. You fight it by holding out test data, using cross-validation, simplifying the model, gathering more data, or applying regularization to keep the model from contorting itself to fit every point.