model compression

A trained vision model is usually far bigger and slower than it needs to be for deployment. Model compression is the umbrella term for the family of techniques that shrink a model's storage size, memory footprint, and compute cost while preserving as much accuracy as possible — so it fits on a phone, meets a latency budget, or simply costs less to serve at datacenter scale.

The four main tools are quantization (fewer bits per number), pruning (remove redundant weights or whole channels), knowledge distillation (train a small student from a big teacher), and low-rank factorization (replace big weight matrices with products of small ones). Adjacent ideas include weight sharing / clustering and entropy coding — the classic Deep Compression pipeline combined pruning, quantization, and Huffman coding for 35–49× size reduction on AlexNet and VGG — and efficient architecture design (MobileNet's depthwise-separable convolutions, EfficientNet's compound scaling), which "compresses by construction" rather than after the fact.

These techniques are complementary and routinely combined: distill to a small network, prune its channels, then quantize to int8 for an NPU. Choose by what is scarce — storage favors quantization and factorization; compute and latency favor structured pruning and efficient architectures; a tight accuracy budget calls for distillation to claw it back. Crucially, compression is co-design with the deployment target: always validate on the actual hardware, because a technique that helps one chip can be useless on another.