New Agent Instrument Verifies Long-Horizon LLM Behavior

Verifying the behavior of long-horizon AI agents is inherently difficult because their internal states and self-reports cannot always be trusted. This research presents a novel agent instrument designed for structural verification, rather than post-hoc analysis, to address this fundamental challenge. The instrument operates by dissociating the roles of a Large Language Model (LLM) and a deterministic "Executive." The Executive maintains all belief and control, while the LLM is restricted to filing typed proposals. A claim is only accepted if a pre-registered prediction matches an observation, verified by code. This architecture ensures that verification is built into the system's design. The instrument also includes self-invalidation mechanisms and a "shadow reference" that compiles an ideal plan for comparison, allowing for precise measurement of "commitment drift" (abandoning goals) versus "binding drift" (errors in executing a plan). The study demonstrates that ablating the commitment mechanism drastically increases goal abandonment while binding error remains flat, highlighting the distinct nature of these failure classes. Although the current agent has null task efficacy on ARC-AGI-3, the contribution lies in its robust verification methodology for agent development.