New AI Algorithm Boosts Multi-AUV Target Tracking in Dynamic Oceans

Tracking maneuvering targets using multiple autonomous underwater vehicles (AUVs) in an ad-hoc network presents significant challenges, including limited acoustic communication, dynamic network topologies, and unpredictable ocean disturbances. Existing multi-agent reinforcement learning (MARL) methods often struggle with high-dimensional state-action spaces and noise-sensitive policy generation, leading to unstable training and suboptimal tracking. This research proposes a novel solution called VGG-MADiffRL, a value-gradient-guided multi-agent diffusion reinforcement learning algorithm, alongside MDCA, a diffusion-based hierarchical control architecture. The MDCA framework establishes a three-tier control loop that optimizes task allocation, local decision-making, and physical execution. Within this, VGG-MADiffRL leverages diffusion policies and incorporates value gradients to steer action generation towards higher expected returns, while twin value networks mitigate overestimation and stabilize training. Experimental results demonstrate that this approach significantly improves convergence speed, tracking accuracy, and training stability in complex underwater environments.