1.12.11
Constrained Decoding: Structured Generation
What is it?
Constrained decoding restricts which tokens a model is allowed to generate at each step. It uses a formal grammar or structure, such as a JSON schema, regular expression or context-free grammar, to keep output valid.
Key points
A parser tracks the current generation state. A token mask identifies which next tokens are allowed. Disallowed tokens receive a negative-infinity logit before softmax. The mask is updated after every generated token. The source describes a pipeline from JSON Schema to regex, finite-state machine and token masks. Libraries include Outlines, lm-format-enforcer, Guidance and XGrammar.
Generating Valid JSON
Suppose an application requires an output with name and price fields. Constrained decoding uses a schema to restrict the available tokens. The model cannot generate an invalid JSON prefix because tokens that violate the required structure are masked at each step.
Real-world application
Useful for API responses, structured extraction, data pipelines and applications that require output to follow a specific schema or grammar.
Key takeaway
Constrained decoding enforces output structure during generation instead of relying only on instructions in the prompt.