receptive field

The receptive field of an output unit is the region of the original input that can influence its value — the 'window onto the world' that one neuron actually sees. A single 3x3 convolution gives each output a 3x3 receptive field: it only knows about a 3x3 patch. But when you stack layers, each new layer looks at a neighbourhood of the previous layer's outputs, and those outputs each already summarized a patch, so receptive fields compound and grow with depth. This is the mechanism by which a deep CNN, built only from tiny 3x3 filters, can eventually 'see' a whole object.

Precisely, for a stack of layers with kernel sizes k, strides s, and dilations d, the receptive field grows layer by layer. With stride 1 and no dilation, stacking L layers of kernel k gives a receptive field of size 1 + L(k-1); for example ten 3x3 layers reach 1 + 10*2 = 21 pixels across. Stride multiplies the growth (each strided layer scales the step at which earlier receptive fields are spaced), and dilation enlarges a layer's footprint without adding parameters, so both let the receptive field expand far faster than depth alone. The recurrence is r_l = r_{l-1} + (k_l - 1) * (product of all strides before layer l), counting dilation by replacing k with its dilated size.

A crucial caveat is the difference between the theoretical and the effective receptive field. Even if a unit could in principle be influenced by a 200x200 region, the central pixels dominate and the influence falls off roughly like a Gaussian toward the edges; Luo et al. (2016) showed the effective receptive field is often only a fraction of the theoretical one and grows merely with the square root of depth. This is why segmentation and detection architectures deliberately add dilation, larger strides, or global pooling to guarantee enough real context, rather than trusting depth alone.