Trie Automata Accelerate LLM Constrained Decoding for Structured Output

Large Language Models (LLMs) are increasingly required to produce structured outputs that adhere to predefined schemas, often involving selection from a vast finite set of valid strings. Current constrained decoding systems struggle with this "cardinality wall" as the number of valid values grows, leading to prohibitively slow performance. This research introduces the trie automaton, a novel mechanism specifically designed to address this challenge. The trie automaton leverages the inherent structure of finite sets, such as shared prefixes and bounded depth, by employing Aho-Corasick multi-pattern matching to precompute per-node token masks. This specialized approach results in a 7X faster per-step valid-token computation compared to existing general-purpose grammar compilation backends like XGrammar, and 2-6.5X faster compilation for larger sets. Crucially, these precomputed masks enable a stateless serving path that bypasses the guided decoding pipeline, leading to compounding advantages in batch serving. End-to-end vLLM throughput reached 219 requests/second with the trie automaton versus 7.5 requests/second with XGrammar at batch size 256, representing a 29X improvement. The trie maintains sub-100ms compilation for up to 10,000 values and offers a flat per-step cost regardless of set size, guaranteeing 100% output validity across various tokenizer families.