quantization-aware training (QAT)

Quantization-aware training lets the model practice being low-precision while it is still learning. During the forward pass it rounds weights and activations onto the target grid, so the network feels the quantization error, but it keeps a full-precision shadow copy of the weights that the optimizer actually updates, letting the model adjust itself to minimize the damage.

The non-differentiable rounding is handled with the straight-through estimator: in the backward pass the round is treated as the identity so gradients flow through unchanged. The quantizer's scale, and sometimes its clipping range, can be learned jointly with the weights, as in LSQ. QAT either trains from scratch or, more commonly, fine-tunes a pretrained model for a short schedule on a fraction of the data.

QAT typically recovers more accuracy than post-training quantization at aggressive settings — INT4 and below, or quantized activations — but it costs data, gradients, and full training infrastructure, so it is reserved for the cases where the cheaper post-training route falls short.