Monte Carlo dropout (MC dropout)

Dropout is normally a training-time regularizer that you switch off at test time. Monte Carlo dropout keeps it on at test time: you run the same input through the network many times, each pass randomly zeroing a different subset of units, and treat the spread of the resulting predictions as a measure of uncertainty. Where the passes agree the model is confident; where they scatter it is unsure.

Gal and Ghahramani showed that training a network with dropout and weight decay is equivalent to variational inference in a deep Gaussian process: each dropout mask corresponds to a sample from an approximate posterior over weights, and averaging T stochastic forward passes is Monte-Carlo integration of the predictive distribution. The predictive mean is the average of the passes; the predictive variance, plus an inverse-precision term, estimates epistemic uncertainty.

It is nearly free, with no retraining required, which is why it is the default cheap uncertainty baseline. But the implied posterior is a crude Bernoulli mixture: its uncertainty does not always shrink correctly as data grows, it is often poorly calibrated, and it depends heavily on the dropout rate, which is fixed by hand rather than inferred from data.