LLMs are Terrible Backends: Forcing Strict JSON Output
When you use LLMs as API endpoints, their probabilistic nature breaks downstream systems. Here is how to enforce strict JSON output through grammar-constrained generation and structured outputs.
Tag: JSON
LLMs are Terrible Backends (Unless You Force JSON)
Non-determinism is a bug, not a feature. We explore how to whip the model into compliance using Enforcers, Pydantic, and Constrained Generation.