CommitKV Compresses KV Cache for Multi-Turn AI Agents

Multi-turn AI agents, particularly those using the ReAct framework, accumulate extensive reasoning trajectories, tool calls, and observations, leading to rapidly growing Key-Value (KV) caches. This expansion increases memory consumption and attention costs during inference. Traditional KV cache compression methods often evict states based on low attention scores, which can inadvertently remove information that is temporarily inactive but crucial for future turns. CommitKV addresses this by introducing a lifecycle-aware compression strategy that leverages "commit transitions." It distinguishes between temporarily dormant information and data that has completed its role by comparing deletion effects before and after tool-call commits and their observations. This allows CommitKV to safely remove irrelevant pages while protecting active or pending information, leading to substantial reductions in memory use and inference time, alongside improved accuracy on various benchmarks.