Adaptive Hybrid PSO Optimizes Gradient Descent Integration

This research presents Adaptive Hybrid Particle Swarm Optimization (AHPSO), an enhanced optimization algorithm that intelligently integrates gradient descent into the traditional Particle Swarm Optimization (PSO) framework. The core innovation of AHPSO is its use of a sigmoid function to dynamically adjust the influence of gradient information based on the swarm's diversity. This means gradient descent is minimally applied during the exploration phase when diversity is high, and maximally applied during the exploitation phase when the swarm has converged on a potential solution basin. The study clarifies that gradient injection is not universally beneficial for PSO but is most effective when the swarm has identified a smooth local structure. AHPSO demonstrates superior performance over standard PSO and other methods like CMA-ES on problems characterized by such smooth local basins, achieving higher rankings in iteration-matched comparisons. This work provides a principled understanding of when and how gradient information adds value to swarm-based search algorithms.