model compression

Model compression is the umbrella term for all the techniques that make a trained model smaller, faster, or cheaper to run without retraining it from scratch. If quantization, pruning, and distillation are individual tools, model compression is the whole toolbox and the craft of using them. The goal is always the same: take a model that works but is too heavy for where you need it, and slim it down enough to fit — onto a phone, into a tight latency budget, or under a smaller cloud bill.

The main tools combine in practice. Quantization stores each number with less precision. Pruning deletes parts that barely matter. Distillation trains a smaller model to imitate a bigger one. There are others — factorizing big weight matrices into smaller ones, sharing weights across layers — but these three are the workhorses, and they are often stacked: distill to a smaller model, prune it, then quantize what remains.

Why it matters: compression is what stands between an impressive research model and one people can actually deploy at scale or on a device. It is one of the most practically important areas of applied ML. The honest theme running through all of it: every method trades some quality for size or speed. Sometimes that loss is negligible; sometimes it lands precisely on the cases you care about. There is no free compression — the engineering skill is in measuring the loss honestly and deciding whether the tradeoff is worth it for your use.