training vs inference
/ TRAY-ning vurs IN-fur-uns /
Every machine learning model lives two very different lives. The first is training: the model studies a pile of examples and slowly adjusts its internal numbers until its answers stop being wrong. The second is inference: training is over, the numbers are frozen, and the finished model is simply put to work answering new questions. It is the difference between a student cramming for an exam and that same student, years later, using what they learned on the job.
The two phases differ in almost every practical way. Training is a one-time (or occasional) effort that is slow, hungry for data, and computationally expensive — it may take weeks on rooms full of specialized chips. Inference happens over and over, ideally in a fraction of a second, every time someone asks the model something. Training needs the right answers (labels) to learn from; inference needs only the question. A model is trained once and may then run inference billions of times.
Why insist on the distinction? Because the costs and risks live in different places. Training is where bias, errors, and overfitting get baked in; inference is where speed, energy use, and serving cost dominate, since you run it constantly. It is also why a deployed model does not keep learning from each query unless it is explicitly retrained — during inference the model is frozen and simply applies what it already knows, mistakes and all.
A face-unlock model is trained once, in a lab, on millions of faces over many days. On your phone it does inference: each time you glance at the screen, the frozen model checks "is this the owner?" in a few milliseconds. It never learns your new haircut from those glances unless an update retrains it.
Train slowly once with answers; infer fast, forever, on new questions.
"Inference" here just means "running the trained model to get a prediction" — not the deeper statistical sense of inference (estimating how the world works). In everyday ML talk, inference is simply the prediction step.