New RL Communication Gating Uses Belief Divergence for Efficiency

Multi-agent reinforcement learning systems often struggle with deciding when agents should communicate, leading to either constant, inefficient communication or unstable learned gating. A new approach suggests that agents should only exchange information when their internal "beliefs" about the world state differ substantially. This is quantified using KL divergence between their belief distributions, which are derived from their LSTM hidden states. This principled gating mechanism was tested on Predator-Prey and MPE simple_spread benchmarks. While it performed comparably or slightly less well on easier tasks, it showed significant improvements on more complex scenarios, such as the larger Predator-Prey environment, achieving higher success rates and fewer steps than previous methods. The research also indicates that the underlying belief representation itself contributes to better coordination, even when the gating mechanism is inactive.