Context Compression for AI Agents Can Cause Instability

Long-horizon AI agents, which operate over extended periods and require maintaining a vast amount of contextual information, often employ context compression techniques to manage the growing size of their operational memory. However, the behavioral implications of such compression have been poorly understood. This empirical study highlights a critical issue: context compression can inadvertently weaken the influence of recent interactions, leading to undesirable behaviors such as an in crease in blocked actions, repetitive exploration, and overall instability across different runs of the agent. Motivated by these observations, the researchers developed TRACE (Testbed for Reliable Agent Context Evaluation), a verifier-guided framework designed to improve the reliability of context compression. TRACE evaluates individual compression events by comparing closed-loop continuations from the same environment state, both with and without compression. It then uses summary preferences derived from these comparisons to optimize the natural-language compression prompt, all while keeping the underlying AI models frozen. Initial results from experiments conducted on the AppWorld environment are promising. TRACE demonstrated improvements over existing compression baselines in several key areas: enhanced task performance, greater multi-run reliability, and better context-execution efficiency. These findings suggest that a boundary-local evaluation approach, as implemented in TRACE, is a viable and promising direction for developing more reliable and stable context compression strategies for long-horizon AI agents.