SmoothQuant
Activations in large transformers contain a few channels with huge outlier values that make them very hard to quantize, while the weights are smooth and well-behaved. SmoothQuant's trick is to mathematically borrow some of that difficulty from the activations and hand it to the weights, so that both sides of every matrix multiply become quantizable to INT8.
It introduces a per-channel smoothing factor s: divide activation channel j by s_j and multiply the corresponding weight row by s_j, which leaves the product Wx unchanged. The factor balances the two dynamic ranges, s_j = max|x_j|^α / max|W_j|^(1−α), with a migration strength α near 0.5 that controls how much difficulty moves over. This is an offline transform fused into the preceding layer, after which the whole layer runs as W8A8 INT8.
Unlike weight-only schemes, SmoothQuant quantizes activations too, so it accelerates the compute-bound matrix multiplies that dominate prefill and large-batch serving, and it is widely deployed for INT8 LLM inference. Its single knob α must be tuned per model family, since the outlier structure differs across architectures.
A per-channel factor s migrates activation difficulty into the weights, keeping the product fixed.
The migration strength α is a slider, not a free lunch: pushing too much difficulty into the weights can hurt weight quantization instead, so the balance point is model-specific.