post-training quantization

You already have a fully trained FP32 model and you do not want to retrain it — no labels, no GPU-days, no training pipeline. Post-training quantization (PTQ) converts that model to low precision directly, using only a small unlabeled "calibration" set of representative images to figure out the right scales. It is the fast, practical path to int8: minutes of work, not a training run.

Weight scales are computed straight from weight statistics (often per-channel min/max). Activation scales need calibration: you run a few hundred representative images, collect a histogram of each activation tensor, and pick a clipping range that minimizes error. Common criteria are plain min/max, a high percentile (clip rare outliers), or KL-divergence / entropy between the float and quantized distributions (TensorRT's classic method). "Static" PTQ bakes these activation scales in; "dynamic" PTQ instead computes each activation's range at runtime per input (no calibration set needed, popular for transformers and NLP).

Plain int8 PTQ on CNNs is near-lossless, but aggressive settings (int4) or transformer/ViT models need smarter methods: AdaRound (learn whether to round each weight up or down instead of round-to-nearest), bias correction (compensate the mean error a layer introduces), cross-layer equalization, plus GPTQ/AWQ for weights and SmoothQuant for migrating activation outliers into the weights. When even these drop too much accuracy, you escalate to quantization-aware training.