constrained decoding

Constrained decoding forces the model's output to obey a formal shape — a grammar, a regular expression, a JSON schema, a list of allowed words. At each step it computes which tokens could possibly continue a valid string and masks out all the rest by setting their probability to zero, so the model can only sample from legal options. The result is output that is guaranteed to parse, every single time, no matter how the model would have wandered on its own.

This turns flaky structure into a hard guarantee, which is why it underpins reliable tool calling, code that must compile, and data pipelines that cannot tolerate a stray bracket. The cost is real: building and updating the allowed-token mask adds overhead, an overly tight grammar can box the model into awkward or wrong content just to stay legal, and constraints control form, not truth — a perfectly valid JSON object can still hold a wrong answer. It is the engine behind JSON mode and grammar-guided generation.