learning curve
/ LUR-ning kurv /
A learning curve is a plot of how the model's performance changes as training proceeds — the single most-watched diagnostic in machine learning. Most often it shows the loss (or accuracy) on the y-axis against training time on the x-axis, for both the training set and a validation set. Reading these two curves together tells you, at a glance, whether training is going well, stuck, or going wrong, long before you waste hours.
The shapes tell stories. If both curves fall and settle close together, training is healthy. If the training curve keeps falling but the validation curve flattens and then rises, you've crossed into overfitting — the model is memorizing. If both curves plateau high and refuse to drop, you're underfitting — the model is too weak, or the learning rate is wrong, or it needs more training. A jagged, exploding curve usually means the learning rate is too high.
There's a second, related sense: a learning curve can also plot performance against the amount of training data, holding training time fixed. That version answers a practical and money-saving question — "would collecting more data actually help, or have we hit the point where more examples barely move the needle?" In both senses, the value of a learning curve is that it turns an invisible process into a picture you can act on.
Two curves on the same plot. The blue training-loss line slides steadily down to 0.05. The orange validation-loss line drops to 0.30 at epoch 20, then visibly climbs back to 0.45. The widening gap after epoch 20 is the unmistakable signature of overfitting — and the cue to stop at epoch 20.
A widening train-validation gap is overfitting made visible.
Always plot the validation curve, not just the training curve. A beautiful, ever-falling training loss tells you almost nothing on its own — the gap to validation is where the truth lives.