alignment tax

The alignment tax is the capability you sometimes give up to make a model safer or better behaved. After heavy safety tuning a model might refuse harmless edge cases, lose a little raw accuracy on hard problems, or become more cautious and verbose. That gap between what the base model could do and what the aligned model will do is the tax.

Some of the tax is genuine and some is avoidable. A model trained to refuse with a blunt template can over-refuse and frustrate users; a model whose preference data over-rewards caution learns to hedge. Better data, clearer principles, and careful objectives shrink the tax, and a common finding is that well-done alignment can be close to free, or even improve usefulness, because following intent is itself a capability.

The phrase is useful as a reminder to measure both sides. Safety gains that quietly gut helpfulness are a bad trade, and so are capability gains that quietly erode safety; the goal is to push the frontier where you get both.