epistemic vs. aleatoric uncertainty

Two very different reasons a model can be unsure. Aleatoric uncertainty is irreducible noise in the data itself: the coin is fair, the sensor is grainy, the label is ambiguous, and no amount of extra data removes it. Epistemic uncertainty is the model's own ignorance: it has seen too little data in this region and would shrink if you collected more. Telling them apart is the whole point of uncertainty quantification.

Aleatoric uncertainty is captured by letting the model predict its own noise, through a heteroscedastic variance head or the spread of the likelihood. Epistemic uncertainty lives in the posterior over parameters: it is the disagreement among plausible models, formally the variance of the predictive that comes from integrating over parameters. A standard decomposition writes total predictive uncertainty (the entropy) as expected aleatoric uncertainty (the conditional entropy) plus epistemic uncertainty (the mutual information between the prediction and the parameters).

The distinction drives decisions: epistemic uncertainty tells active learning where to gather data and tells out-of-distribution detection when to abstain, while aleatoric uncertainty tells you the floor on achievable error. The caveat is that the split is model-relative: what looks aleatoric to a small model may be reducible by a better one, and many cheap methods conflate the two.