pruning

Pruning makes a trained model smaller by deleting the parts that barely contribute — the connections or neurons whose removal hardly changes the answers. The name comes from gardening: a tree often grows healthier and tidier once you cut away the dead and redundant branches. Neural networks are similarly overgrown; trained with far more connections than they end up needing, so a large fraction can be snipped away with little loss.

In practice, you score each weight or neuron by how much it matters — often just by how small it is, on the idea that near-zero weights barely affect anything — then set the least important ones to zero or remove them entirely. Usually you then fine-tune the trimmed model briefly to let it recover, much as a pruned plant regrows. The result is a model with fewer parameters that is cheaper to store and, ideally, faster to run.

Why it matters: pruning can shrink models substantially, and it pairs naturally with quantization for deployment on limited hardware. But there is an honest catch that newcomers miss: removing scattered individual weights (unstructured pruning) makes a model that is smaller on paper but often no faster in practice, because ordinary hardware can't skip the zeros efficiently — it still does the multiply-by-zero. Real speedups usually require structured pruning, which removes whole chunks the hardware can genuinely skip, and that tends to cost more accuracy. Pruning is useful but its benefits are easy to overstate.