Graph Feedback Influences LLM Consensus and Cliques

Multi-agent language model systems are becoming increasingly common, yet the underlying runtime interaction graph is often overlooked as a critical factor. This research delves into how this graph influences the formation of conventions and the emergence of consensus or fragmentation within populations of open-weight language models, ranging from 1.1B to 32B parameters. The study employs a naming-game protocol, using restricted first-token scores to measure prompt-conditioned score-state distributions and construct state-similarity graphs. This allows for distinguishing between sampled-label agreement and latent state-space consensus. Across various controlled interventions, the findings indicate that retaining partner-label evidence is necessary but not sufficient for achieving consensus. Specifically, homophilous threshold-similarity routing, which prioritizes interactions with similar models, tends to delete cross-basin exposure and amplify fragmentation within the population. In contrast, bridge-seeking routing, which actively seeks connections across different "basins" of models, often succeeds in repairing fragmentation, especially when memory is available to the agents. In a mixed four-model grid, threshold-similarity routing consistently failed to produce behavioral or state consensus, whereas state-component and label-disagreement bridges successfully recovered final behavioral consensus in a significant majority of retained-memory runs. For homogeneous model populations, retaining historical interactions generally shifted fragmented dynamics towards consensus, with Qwen2.5-32B being a clear example. The qualitative ordering of these effects remained robust across variations in state thresholds, population size, and vocabulary size, with early-window graph-energy features providing useful diagnostic insights.