OPERNEX LEARNING

Learn Agentic AI

A structured, practical learning path from language model foundations to intelligent agentic systems.

6 Learning parts
453 Subtopics
415 Published lessons
Foundations / LLM Architecture and Optimization Methods / Text Generation: Decoding Methods
Lesson 77 of 415 7 min read
1.12.11

Constrained Decoding: Structured Generation

What is it?

Constrained decoding restricts which tokens a model is allowed to generate at each step. It uses a formal grammar or structure, such as a JSON schema, regular expression or context-free grammar, to keep output valid.

Key points

A parser tracks the current generation state. A token mask identifies which next tokens are allowed. Disallowed tokens receive a negative-infinity logit before softmax. The mask is updated after every generated token. The source describes a pipeline from JSON Schema to regex, finite-state machine and token masks. Libraries include Outlines, lm-format-enforcer, Guidance and XGrammar.

Generating Valid JSON

Suppose an application requires an output with name and price fields. Constrained decoding uses a schema to restrict the available tokens. The model cannot generate an invalid JSON prefix because tokens that violate the required structure are masked at each step.

Real-world application

Useful for API responses, structured extraction, data pipelines and applications that require output to follow a specific schema or grammar.
Key takeaway

Constrained decoding enforces output structure during generation instead of relying only on instructions in the prompt.