New Benchmark Evaluates LLM Agents in Complex Bargaining Games

Researchers have introduced SidConArena, a novel benchmark environment specifically designed to assess the capabilities of large language model (LLM) agents in complex, open-ended, and positive-sum bargaining scenarios. Unlike traditional static reasoning or zero-sum games, SidConArena models a multi-player economy where agents must negotiate, create shared value, compete for assets, and plan for future returns. The framework incorporates natural-language negotiation, deterministic production processes, and sealed-bid auctions for long-term assets, all within a partially observable stochastic game. Evaluations conducted in homogeneous and heterogeneous tournaments showed that advanced LLMs achieve superior economic outcomes. However, these agents still exhibit limitations, such as misvaluing resources, passive bargaining strategies, and difficulties with long-horizon investment planning, indicating areas for further improvement in agentic AI.