invariant risk minimization (IRM)

Standard training rewards any feature that lowers error, including spurious shortcuts like a background that happens to correlate with the label. Invariant risk minimization tries to immunize against such shortcuts by demanding that the model rely only on features whose relationship to the label stays the same across different training environments, the intuition being that a stable, causal relationship will hold everywhere, while a spurious one will shift when the environment does.

Concretely, IRM seeks a data representation such that a single linear predictor on top of it is simultaneously optimal in every environment. Because that bilevel condition is hard to optimize, the practical objective IRMv1 adds a gradient-penalty term that pushes the per-environment optimal classifier toward being the same constant, alongside the usual average risk. Features that are predictive in one environment but not invariant across them are thereby penalized, in principle leaving a predictor that leans on causal rather than incidental structure.

IRM is a landmark attempt to operationalize the causal idea that mechanisms are invariant and confounds are not, connecting it to out-of-distribution generalization and domain robustness. It is also actively contested: follow-up work shows the relaxed objective can fail to recover the invariant predictor, can need impractically many or diverse environments, and is sometimes beaten by careful empirical risk minimization, so it is better seen as a fertile research direction than a solved recipe.