resnet

ResNet (Deep Residual Network, He, Zhang, Ren, and Sun at Microsoft Research, 2015) is the architecture that took the residual-connection idea and used it to win ILSVRC 2015 with networks 8 times deeper than VGG yet still trainable. It demonstrated that with residual blocks you can build networks of 50, 101, 152, and even over 1000 layers, and that, contrary to the earlier degradation problem, deeper genuinely meant better. ResNet swept the 2015 ImageNet classification, detection, and localisation tasks, and it remains, in its many variants, one of the most widely deployed backbones in all of computer vision a decade later.

The atom of ResNet is the residual block: two or three convolutional layers whose output is added to the block's input via a skip connection, with batch normalization after each convolution and ReLU activations. The deeper ResNets use a bottleneck block, a 1x1 convolution that reduces channels, a 3x3 convolution that does the spatial work on the reduced representation, and a 1x1 convolution that restores the channel count, with the skip connection added around all three. This bottleneck (again the Network-in-Network 1x1 trick) keeps the computational cost of very deep models manageable. The network is organised into a few stages; at the start of each stage spatial resolution halves and channel count doubles, and a projection shortcut handles the shape change.

ResNet's importance is hard to overstate. It made depth a freely usable resource, so that practitioners could scale networks without fear of the optimisation collapsing. Its design choices, batch normalization plus residual blocks plus stage-wise downsampling, became the template that ResNeXt, DenseNet (a contrast variant), SE-ResNet, and many detection and segmentation systems built upon. Pretrained ResNet-50 in particular became the de facto standard feature extractor for transfer learning across the field. Even in the transformer era, ConvNeXt explicitly modernises a ResNet to match Vision Transformers, and ResNet-style residual stages still appear inside detectors, segmenters, and diffusion backbones.