mean squared error

Mean squared error (MSE) is the workhorse loss for regression, predicting continuous numbers rather than classes. You take the difference between each prediction and its target, square it (so over- and under-shooting both count as positive error, and big mistakes count disproportionately), and average over all predictions. The squaring is the defining choice: an error of 4 is penalized 16, four times as much as an error of 2 is penalized 4, so MSE cares intensely about large deviations.

For predictions y_hat and targets y over N values, MSE = (1/N) times the sum over i of (y_hat_i - y_i)^2. It is the squared Euclidean (L2) distance between prediction and target, averaged. Its gradient with respect to each prediction is 2(y_hat_i - y_i)/N, linear in the error, so the correction grows with how wrong you are. Minimizing MSE is maximum-likelihood estimation under the assumption that targets equal the model's output plus Gaussian noise, and the value that minimizes expected squared error is the mean of the target distribution, which is why MSE-trained regressors predict conditional means.

MSE is smooth and convex in the prediction, giving clean, stable gradients. But its sensitivity to large errors makes it fragile to outliers, and predicting the mean makes it blur multi-modal outputs: an image-reconstruction or prediction model trained on pure MSE produces characteristically blurry results, because the safe bet under squared error is to average all plausible answers. Alternatives trade these off: mean absolute error (L1) predicts the median and is more robust and sharper but has a constant gradient; Huber or smooth-L1 is L2 near zero and L1 in the tails, combining stability with outlier-resistance.

MSE and L1 are used for continuous targets and reconstruction throughout vision: image super-resolution and denoising, depth and optical-flow regression, keypoint and bounding-box coordinate regression, and the pixel reconstruction term in autoencoders. Crucially, the diffusion models behind modern image generation are trained with a mean-squared-error objective (predicting the noise added to an image), showing MSE is alive at the frontier. To beat MSE blur, generators add perceptual or adversarial losses on top.