ASALT Improves Multi-Agent RL Transfer with Adaptive State Alignment

Multi-agent reinforcement learning (MARL) often struggles with transferring learned policies between different environments, especially when the observation or global state spaces have varying dimensionalities. Existing transfer learning methods typically require these spaces to be identical, limiting their applicability in heterogeneous scenarios. Researchers have introduced ASALT (Adaptive State Alignment for Lateral Transfer), a novel approach that explicitly handles these mismatched dimensionalities. ASALT employs both observation-level and state-level adapters. These adapters function by mapping the distinct observations and global states from a target domain into a common embedding space, facilitating more effective knowledge transfer for both the actor and critic components of the MARL system. Experiments conducted in standard benchmark environments demonstrate that ASALT surpasses current baselines in terms of sample efficiency and overall global return within cooperative settings. The effectiveness of ASALT is shown to depend on the degree of mismatch between the source and target domains, and it successfully mitigates negative transfer, a common challenge when transferring policies across diverse environments.