OGR-MARL Boosts USV Cooperative Pursuit in Constrained Waterways

Cooperative pursuit by heterogeneous Unmanned Surface Vehicles (USVs) in constrained environments like port waterways presents significant challenges due to navigation, traffic, and role-specific constraints. Traditional multi-agent reinforcement learning (MARL) often struggles to learn effective policies from scratch in such complex settings. Researchers have proposed OGR-MARL (Option-Guided Residual Multi-Agent Reinforcement Learning), a framework designed to overcome these difficulties. OGR-MARL decouples from specific MARL algorithms, instead integrating shared evader belief, role-conditioned option targets, adaptive rule penalties, and residual policy learning. This allows MARL algorithms to learn corrective actions on top of pre-defined rule-guided behaviors, rather than exploring the entire constrained environment from scratch. The framework was instantiated with several continuous-control MARL backbones (e.g., MADDPG, MASAC). Experiments in an abstract port scenario demonstrated that the OGR-MASAC instantiation achieved a 75.0% capture rate, showcasing effective rule compliance and superior heterogeneous coordination. Furthermore, zero-shot transfer to a more complex, real-world-informed map yielded promising results, indicating OGR-MARL's strong generalization potential for practical deployment.