empirical risk minimization
/ em-PEER-ih-kul RISK min-ih-mih-ZAY-shun /
Empirical risk minimization is the quiet principle underneath almost all machine learning: since you can't measure how well a model will do on every situation in the world, you settle for measuring how well it does on the examples you actually have, and you make that as good as possible. "Empirical" means "based on the data in front of you"; "risk" is just a formal word for average loss; "minimization" means push it as low as you can.
The honest backstory is that what we truly want — low error on all future, unseen cases (the "true risk") — is unmeasurable, because the future hasn't happened and the world is infinite. So we substitute a stand-in we can compute: the average loss over the training set (the "empirical risk"). ERM is the bet that doing well on a large, representative sample will translate into doing well in general. Often it does — but only if the sample really looks like the world the model will face.
This single idea is also where overfitting comes from. A model with enough flexibility can drive the empirical risk to zero by memorizing every training example, including its noise and quirks, while the true risk stays high. That gap is exactly why we don't trust training loss alone, why we hold out a validation set, and why techniques like regularization and early stopping exist — they keep ERM honest by discouraging the model from fitting the sample too perfectly.
You collect 10,000 labeled emails and train a spam filter to minimize average error across exactly those 10,000. That average is the empirical risk. What you really wanted was low error on the millions of emails you'll get next year — the true risk you can never directly measure. ERM hopes the 10,000 are a faithful stand-in.
We minimize the loss on data we have, hoping it stands in for data we don't.
Minimizing training loss is not the goal — generalizing is. ERM only works when the training data is large and representative; on a biased or tiny sample, perfectly minimizing empirical risk can make the model worse on the real world, not better.