R2D-RL: New RoboCup 2D Environment for MARL Research

Multi-agent reinforcement learning (MARL) research often benefits from complex, dynamic environments that mimic real-world challenges. Robot soccer, particularly the RoboCup 2D Soccer Simulation (RCSS2D), serves as an excellent testbed due to its inherent complexities, including partial observability, a mix of cooperative and adversarial interactions, sparse reward signals, and the need for long-horizon tactical planning. However, integrating RCSS2D's competition-oriented server-client architecture directly into contemporary Python-based MARL workflows has historically been difficult. To address this, R2D-RL has been introduced as a new reinforcement learning environment. R2D-RL facilitates this connection by linking RCSS2D and HELIOS-based player clients to a Python MARL interface through shared-memory communication and cycle-level synchronization. It supports various training scenarios, configurable opponents, different action spaces, reward shaping, and parallel execution, providing a robust platform for MARL research and development.