Structured Generation

Structured generation produces outputs in specific formats (JSON, XML, code) with guaranteed validity. Problem: LLMs sometimes produce invalid formats despite instructions - malformed JSON, syntax errors, schema violations. Solution: Constrain token selection to only valid continuations during decoding. Approaches: Grammar-constrained: Define format grammar, reject invalid tokens at each step. Schema-guided: JSON Schema or Pydantic models specify structure, generate compliant outputs. Template-based: Fill in designated slots in predefined structure. Tools: Outlines (fast grammar-guided generation), Instructor (Pydantic-based extraction), Marvin, Guidance, llama.cpp GBNF grammars. JSON example: Define schema → during generation, only allow valid JSON tokens → output guaranteed parseable. Performance: Minor latency overhead, major reliability improvement - eliminates format-related retries. Best practices: Define strict schemas, validate outputs anyway (defense in depth), handle edge cases in schemas. Advanced: TypeScript/Python type generation, nested object extraction, union types. Critical for production pipelines requiring reliable structured data extraction.