1x1 convolution

A 1x1 convolution uses kernels that are one pixel wide and one pixel tall. With no spatial neighbourhood to look at, it does only one thing: at each position independently, it takes the vector of channel values there and produces a new vector by a learned linear combination. In other words it is a tiny fully-connected layer applied identically to every pixel, mixing and re-projecting channels while leaving height and width untouched. Think of it as 'recoloring' each pixel's feature vector.

Precisely, a 1x1 conv with C_in input and C_out output channels holds a weight matrix of shape (C_out, C_in) (plus biases) and computes, at every spatial location, y = W x where x is the C_in-vector of input features at that location. Its cost is H*W*C_in*C_out multiply-adds and its parameters number C_in*C_out — far cheaper than a 3x3, which is 9 times as many parameters and FLOPs. Crucially it can change the channel count: shrink it (a bottleneck) or grow it (an expansion).

This makes 1x1 convolution a workhorse of efficient architectures. Network in Network introduced it; GoogLeNet/Inception use it to cut channels before expensive 3x3 and 5x5 branches; ResNet's bottleneck block sandwiches a 3x3 between a 1x1 that reduces channels and a 1x1 that restores them, cutting compute dramatically; and depthwise separable convolutions pair a depthwise spatial step with a 1x1 pointwise step that does all the cross-channel mixing. Whenever you need to adjust depth, fuse channel information, or insert a cheap nonlinearity-bearing layer, the 1x1 conv is the tool.