Scaling Multi-Agent AI Systems for Enterprise Operations

Enterprise AI systems are increasingly moving towards continuous event monitoring and action across numerous specialized agents. However, current multi-agent systems are often designed for discrete request-response interactions and have not been thoroughly tested or optimized for the complexities of enterprise-level scale. Researchers evaluated two prominent multi-agent architectures, DAG Plan and Execute and ReAct, across 208 real-world enterprise scenarios. These scenarios ranged from small-scale 'Persona' deployments (under 10 agents) to large-scale 'Enterprise' environments (up to 200 agents). The study revealed that the primary challenge to orchestration performance at scale is not task complexity, but rather the noise associated with agent discovery and coordination. Both architectures performed well at smaller scales but experienced significant degradation as the number of agents increased. To address these scaling issues, the researchers introduced a novel Task Manager designed for continuous operation. This manager incorporates features like priority inference, related-event merging, and preemption. The Task Manager demonstrated substantial improvements at enterprise scale, reducing high-priority queue latency by 14-75% and enhancing the correctness of related-event processing by over 20 percentage points. The findings suggest that while DAG Plan and Execute offers precision at smaller scales, ReAct proves more robust in handling failures incrementally, and a dedicated Task Manager is crucial for effective large-scale multi-agent deployments.