resnext

ResNeXt (Xie, Girshick, Dollar, Tu, and He, 2017) refines ResNet by adding a new, orthogonal way to make a network more powerful. The two classic axes for scaling a network are depth (more layers) and width (more channels per layer). ResNeXt introduces a third axis it calls cardinality: the number of parallel transformation branches inside each block. Instead of one bottleneck doing all the work, a ResNeXt block splits the computation into many (for example 32) identical, lower-width branches that each perform a small bottleneck transformation, and then sums their outputs together before adding the residual shortcut. The paper's striking empirical finding is that increasing cardinality is a more effective use of a fixed computational budget than making the network deeper or wider.

This is the same split-transform-merge philosophy as the Inception module, but with a crucial simplification: in Inception every branch is different and hand-designed, whereas in ResNeXt every branch is identical, so the block is defined by just two numbers, the cardinality and the per-branch width. That uniformity makes ResNeXt easy to design, scale, and reason about while retaining the multi-branch benefit. Conceptually each block computes a sum of many aggregated transformations of the same input, which is why the paper is titled 'Aggregated Residual Transformations'.

The elegant implementation detail is that a set of identical parallel branches is exactly a grouped convolution. A grouped convolution partitions the input channels into G groups and convolves each group separately, which is computationally identical to running G independent branches and concatenating, so ResNeXt blocks can be implemented as a single efficient grouped 3x3 convolution sandwiched between 1x1 convolutions. This made grouped convolution a mainstream tool again (it had appeared in AlexNet only as a GPU-memory workaround), and the cardinality idea influenced later efficient designs. ResNeXt backbones became popular in detection and segmentation systems where their accuracy-per-FLOP advantage matters.