Consilience Improves Multi-Agent LLM Reasoning and Communication

This research introduces Consilience, an innovative inference-time orchestration framework designed to optimize communication and reasoning in multi-agent Large Language Model (LLM) systems. It specifically targets "hidden-profile" scenarios where individual agents possess only a portion of the evidence needed to make a correct decision, a common challenge in collaborative AI. Consilience's core mechanism involves summarizing the ongoing discussion at each turn, capturing key aspects like uncertainty, disagreement, evidence gain, and potential premature consensus. Based on this state, it intelligently selects both a communication intervention (e.g., challenge, clarify, seek evidence) and the appropriate agent to speak. A central contribution is a round-wise conformal calibration procedure that provides a statistical guarantee: at each step, the proposed action's regret is bounded with a high marginal probability. This mechanism ensures that conversational actions are appropriate and reliable. Experiments on HiddenBench-style tasks showed that Consilience significantly improves decision accuracy and communication efficiency compared to fixed or unstructured discussion protocols, even outperforming a full-information baseline in some cases, demonstrating the power of certified adaptive communication control.