loss function
/ LOSS FUNK-shun /
A loss function is the single number that tells a learning model how badly it just did. Imagine teaching someone to throw darts: after every throw you measure how far the dart landed from the bullseye. That distance is the loss. A throw dead-center scores zero loss; a wild miss scores a big one. The whole point of training is to nudge the model so this number keeps shrinking.
More precisely, a loss function takes the model's prediction and the true answer and returns a number measuring the gap between them. Different jobs use different yardsticks. For predicting a quantity (a house price), a common choice is squared error — take the difference, square it, so big mistakes hurt much more than small ones. For sorting things into categories (cat vs. dog), a common choice is cross-entropy, which punishes the model for being confidently wrong. The loss is what backpropagation differentiates to find which direction to adjust each weight.
Choosing the loss is one of the most consequential decisions in a project, because the model will optimize exactly what you measure — no more, no less. If your loss rewards a self-driving car for smooth steering but never penalizes hitting a curb, it will learn smooth steering into curbs. The loss is the model's definition of "good," so a sloppy or mismatched loss quietly teaches the wrong lesson, even when every line of code is correct.
A model predicts a house will sell for $300,000; it actually sells for $350,000. Squared-error loss is (350−300)² = 2,500 (in thousands²). A second house: predicted $400,000, sold $410,000, loss (410−400)² = 100. The first mistake counts 25× more, so training will work hardest to fix the big miss.
Squared error makes big mistakes dominate — by design.
The loss is a proxy for what you actually care about, not the thing itself. A model can drive its loss to near zero and still be useless if the loss doesn't capture real-world goals — the famous gap between "low training loss" and "actually good."