pyramid scene parsing

Pyramid Scene Parsing Network (PSPNet) is a semantic segmentation model built on a single observation: many labeling mistakes come from missing the big picture. A network might call a boat a car because, looking only locally, the shape is ambiguous — but if it knew the boat sits on water, the right answer is obvious. PSPNet (Zhao et al., 2017) fixes this by explicitly aggregating context at several spatial scales, including the whole-image scale, before making per-pixel decisions.

Its key component is the pyramid pooling module. Take the feature map produced by a backbone (a dilated ResNet). Pool it down to several coarse grids in parallel — for example 1×1 (a single global descriptor of the entire scene), 2×2, 3×3, and 6×6 (progressively more local sub-regions). Each pooled grid passes through a 1×1 convolution to reduce channels, is upsampled back to the original feature-map size, and all of them are concatenated onto the original features. The result is a representation where every pixel now carries summaries of its sub-region, its larger region, and the whole image at once.

This multi-scale global context is what distinguishes PSPNet from DeepLab's ASPP: ASPP probes context with parallel dilated convolutions (varying receptive field at fixed pooling), whereas PSPNet probes it with parallel pooling to fixed grid sizes (varying spatial summarization). PSPNet won the ImageNet Scene Parsing Challenge 2016 and set strong results on ADE20K and Cityscapes, and its pyramid pooling idea became a standard building block for context aggregation.