Network Topology Shapes Cooperation in Multi-Agent AI Systems

Research into multi-agent reinforcement learning systems often explores how agents learn to cooperate. This study specifically delves into the impact of network topology and the type of information agents receive about their opponents on the emergence of cooperative behaviors. Unlike previous work that often focused on strategy imitation based solely on cumulative payoffs, this paper examines agents learning to play the two-player Iterated Prisoner's Dilemma (IPD) using deep reinforcement learning. In this setup, each agent is represented as a node within a graph, interacting with its neighbors as opponents. During each IPD episode, agents are provided with varying types of opponent information, including action history and opponent identity. Experimental results across different graph topologies reveal that the number of neighbors per node and the average path length within the network are the primary determinants influencing whether cooperation emerges. The study also found that while mechanisms like partner selection can foster mutual cooperation by limiting the diversity of the opponent pool, providing agents with explicit opponent identity information actually hinders the proliferation of cooperative strategies. This suggests that too much specific information about an opponent might lead to less generalized cooperative behavior.