early stopping
/ UR-lee STOP-ing /
Early stopping is the discipline of quitting training while you're ahead. As a model trains, its error on the training data keeps dropping — but at some point it stops genuinely learning and starts memorizing the specific examples, including their noise. Early stopping watches a separate validation set and halts training the moment performance there stops improving, even though the training error would keep falling if you let it run.
The mechanics are simple and cheap. You set aside a validation set the model never trains on, and after each epoch you check the validation loss. While it keeps dropping, keep going. When it bottoms out and begins to rise — the telltale onset of overfitting — you stop, and you keep the version of the model from the best validation point rather than the final one. A small grace period ("patience") is usually allowed, so a single noisy uptick doesn't trigger a premature halt.
It's one of the most practical regularization tools there is: it costs almost nothing, requires no change to the model, and directly targets overfitting by simply not training past the point of diminishing returns. The catch is that it depends entirely on having a trustworthy validation set — if that set isn't representative, or if you tune too aggressively against it, you can stop at the wrong moment or quietly overfit to the validation data instead.
Over 50 epochs, training loss falls steadily the whole way. Validation loss falls until epoch 18, then starts creeping back up. Early stopping keeps the epoch-18 model and discards the rest — the later epochs were just memorizing training quirks that hurt real-world performance.
Stop when validation loss bottoms out, not when training loss does.
Early stopping only works if you decide "best" by validation performance, never by training loss — training loss almost always keeps falling, right past the point where the model has started overfitting.