BitNet (ternary weights)

BitNet pushes quantization to the extreme by training transformers whose weights take only three values: minus one, zero, and plus one. Because three states carry about 1.58 bits of information, the log base two of three, the approach is dubbed 1.58-bit. With such weights the costly weight-by-activation multiplies in a linear layer collapse into additions and subtractions, removing most of the multiplication hardware.

Unlike post-training quantization, BitNet trains the ternary weights from scratch with quantization in the loop: weights are ternarized in the forward pass using an absolute-mean scaling, gradients flow through with the straight-through estimator, and activations are kept at 8 bits. A drop-in BitLinear layer replaces the usual linear layer. In this regime a ternary model can match a full-precision transformer of the same size on language modeling while using far less memory and energy.

The promise is hardware that does away with floating-point matmuls for inference, but realizing the gains needs custom kernels or chips designed for ternary weights. On commodity GPUs without that support, much of the theoretical advantage is left on the table.