Workload-Aware Caching Boosts Multi-Agent System Efficiency

Multi-agent systems break down complex tasks into directed acyclic graphs (DAGs) of specialized agent executions. This structure naturally creates opportunities to cache intermediate results, which can be reused across different queries. However, current cache eviction policies typically treat all cached entries uniformly, relying only on access history and ignoring the unique structural and workload signals available in agentic environments. This paper introduces a novel workload-aware eviction policy designed specifically for multi-agent systems. This policy integrates three crucial signals: the cost of recomputing a result, the number of DAG dependencies an entry has, and the frequency with which an agent is invoked. By combining these into a unified scoring function, the system can intelligently retain the most valuable entries under memory constraints. Evaluated across three multi-agent benchmarks with varying reuse patterns, the new policy demonstrated significant latency reductions. It decreased latency by up to 64.7% compared to an uncached baseline and achieved an average 31.1% latency reduction over the next best finite-capacity baseline. The policy also approached the performance of an unbounded cache while maintaining or exceeding the accuracy of competing finite-capacity methods. The research further highlights that this workload-aware content caching complements other optimization techniques, such as plan-level caching and parallel agent execution, by addressing distinct efficiency bottlenecks within multi-agent pipelines.