Stateful CARS Improves LLM Agent Policy Compliance

This paper introduces Stateful CARS, a novel approach designed for tool-using language model (LLM) agents that operate under complex constraints. These constraints often change dynamically based on observations and prior actions, making it challenging to ensure that an agent's generated actions always comply with the defined policies. Stateful CARS aims to achieve exact sampling from the model's distribution, but critically, it conditions this sampling on a hard stateful validator. The core innovation of Stateful CARS lies in its ability to reuse "invalidity certificates" across different histories. This means that once a particular sequence of actions or a state-continuation schema is identified as invalid, that knowledge can be applied to other similar trajectories, preventing the agent from exploring known invalid paths repeatedly. The method freezes a bank of sound state-continuation schemas within each attempt and removes any trajectory containing a certified invalid continuation at a matching abstract state. The researchers provide theoretical proofs for the method's soundness, adaptive exactness, and other properties. On enumerable workflows, Stateful CARS demonstrated remarkable precision, matching the valid conditional to an extremely high degree of accuracy, even for very low validity probabilities. However, a direct comparison with observation-keyed CARS showed that Stateful CARS was not necessarily cheaper in terms of sampler steps, and the benefits of cross-history transfer were only evident in specific internal ablations. This suggests that while it excels at exact schema-induced conditioning, it doesn't offer a generic systems advantage over existing CARS methods in all aspects.