AI Models for Real Engineering: Decision Models and Constrained Decoding

Filling in a JSON schema, composing an SQL query, classifying an e-mail, rating a document against a rubric: these are examples of small, self-contained tasks that AI models perform in production systems. For such cases, parsing the output of an unconstrained LLM is poor engineering practice. In this workshop, we introduce two better approaches that keep AI-powered systems lightweight and token-efficient. The first part covers Decision Models: models that output a yes/no decision or a calibrated probability distribution over a fixed set of options. The second part covers Constrained Decoding: a technique that guarantees the LLM’s output follows a given format, such as a JSON schema, a regular expression or a grammar. We focus on benchmarking both approaches against baselines (naive LLM prompting, fine-tuned models) in realistic setups. Participants will run the code directly on their laptops using open-weight models and open-source libraries.