structured pruning

The trouble with deleting scattered individual weights is that the weight matrix keeps its shape — it is just full of zeros, and ordinary hardware still multiplies by them. Structured pruning instead removes whole units: entire convolution filters, channels, attention heads, even whole layers. The result is a genuinely smaller, dense network that runs faster on any hardware, with no special sparse kernels or sparsity-aware runtime required.

Concretely, removing an output filter of a conv layer deletes its output feature map, which in turn lets you delete the matching input channel of the next layer — so a whole slice of compute disappears. Importance is scored per filter/channel: the L1/L2 norm of the filter, or the batch-norm scale γ (Network Slimming applies L1 regularization to the BN scales, then prunes channels with small γ). You must respect structural dependencies — residual connections force you to prune the same channels across the layers they join. What comes out is a smaller dense model that cuDNN and ordinary backends accelerate directly.

The tradeoff versus unstructured pruning is that, for a given accuracy, you can remove less (you are deleting coarse units, not cherry-picking individual weights), but the speedup is real and portable. It is usually followed by fine-tuning. Modern automated variants search channel counts directly (AMC uses reinforcement learning, NetAdapt uses empirical latency measurements) or treat layer width as a neural-architecture-search dimension, optimizing for measured latency rather than raw sparsity.