LLM Agents Struggle with Rational Negotiation in Multi-Agent Systems

As multi-agent LLM systems become more prevalent, utilizing protocols like Anthropic's Model Context Protocol (MCP) and Google's Agent2Agent (A2A) for interoperation, questions arise about the strategic correctness of these interactions. These protocols primarily handle transport and discovery, not guaranteeing efficient or rational outcomes. Researchers have introduced a framework to bridge this gap, encoding classical negotiation mechanisms like alternating-offers bargaining and Vickrey-Clarke-Groves auctions as constraints over A2A message schemas. This framework includes a lightweight runtime verification and repair layer to check messages against protocol invariants and a benchmark of negotiation tasks with known optimal solutions. Evaluations across various LLM backbones, using unstructured dialogue, structured protocols, and structured protocols with verification, revealed significant insights. While structured protocols achieved 100% success in negotiation trials, verification further reduced outcome variance. However, mechanism-level incentive compatibility did not automatically translate to rational LLM agent behavior. In auction experiments, both models achieved 100% efficient allocation, but their truthful bidding varied drastically, with one model bidding truthfully in every trial and another in only 3.3% of trials. This highlights that while LLMs can follow structured protocols, their inherent strategic rationality, particularly in complex scenarios like fair allocation, remains a challenge.