conditional random field refinement

Conditional random field (CRF) refinement is a classical post-processing step that takes a neural network's soft, slightly blurry label map and sharpens it so that boundaries snap to the real edges in the image. The intuition is a built-in prior about how the world looks: neighboring pixels that have similar color and are close together almost always belong to the same object, so their labels should agree. A raw CNN, especially the early downsample-heavy ones, ignores this and produces fuzzy, leaky borders; the CRF pulls them back into line.

Formally, a CRF defines an energy over the whole label assignment with two kinds of terms. The unary term is the cost of giving a pixel a particular label, taken directly from the network's per-pixel class scores (it says "the net thinks this is probably road"). The pairwise term is the cost of two pixels disagreeing, and it is large when the two pixels are similar in position and color (it says "these two look alike and are adjacent, so penalize labeling them differently"). Minimizing the total energy trades the network's confidence against spatial-color smoothness.

The version that mattered for deep learning is the fully connected DenseCRF of Krähenbühl and Koltun (2011), in which every pixel is connected to every other pixel through Gaussian kernels over position and color. A naive solve would be intractable, but an efficient mean-field approximation makes it fast, and the whole thing can even be unrolled as recurrent network layers (CRF-as-RNN) and trained end to end. DeepLabv1 and v2 used DenseCRF to gain several mIoU points. It was dropped from DeepLabv3 onward, because stronger backbones, atrous convolutions, and ASPP produce boundaries crisp enough that the extra step stopped paying off.