weight sharing

Weight sharing is the idea that the same small set of kernel weights is reused at every spatial position instead of learning separate weights for each location. A fully-connected layer treats the pixel at row 3 column 5 as completely unrelated to the pixel at row 100 column 200, with its own private weights. A convolutional layer says the opposite: a vertical edge is a vertical edge wherever it appears, so let one detector — one set of weights — scan the whole image. Sharing encodes the prior that useful visual patterns are the same everywhere.

Precisely, in a convolutional layer a single filter of C_in times k times k weights is applied identically at all H times W output positions. This is what makes the parameter count C_out times C_in times k times k — independent of image size — rather than the astronomically larger count a fully-connected layer over the same pixels would need. For a 224x224x3 image, one fully-connected layer to a 224x224 hidden map would need billions of weights; a 3x3 conv with the same channels needs a few hundred. Weight sharing is therefore the single biggest reason CNNs are tractable and data-efficient on images.

Weight sharing has a second, deep consequence: it is exactly what gives convolution its translation equivariance. Because the identical operation runs at every location, shifting the input simply shifts the output — there is no position that is treated specially (boundary effects aside). This is a deliberate architectural inductive bias matched to the statistics of natural images. The trade-off is that pure weight sharing assumes spatial stationarity, which can be suboptimal when position genuinely matters (e.g. faces are usually centred); variants like locally-connected layers, CoordConv, or position embeddings relax it when needed.