ChopChop: Semantically Constraining the Code Output of Language Models

Language models (LMs) can generate code, but cannot guarantee its correctness—producing outputs that often violate type safety, program invariants, or semantic equivalence. Constrained decoding offers a solution by restricting generation to programs that satisfy desired properties. Yet, existing methods are limited to shallow syntactic constraints or rely on brittle, ad hoc encodings of semantics over token sequences. We present ChopChop, the first programmable framework for semantic constrained decoding, enabling LMs to generate code that provably satisfies rich semantic properties. ChopChop enables construction of constrained decoders that incorporate advanced formal methods by connecting token-level generation with reasoning over abstract program structures. It is the first system capable of constraining an LM to only generate programs that are provably equivalent to a supplied reference program. We also show that it can naturally implement existing applications, such as type-constrained decoding for TypeScript.