RL Framework for Zero-Sum Games in Dynamic Environments

Traditional stochastic differential game (SDG) frameworks often struggle with real-world complexities like inaccurate parameter assumptions and abrupt environmental shifts. To overcome these limitations, a new approach is proposed that uses distributional control to define optimal strategies as probability distributions, conditioned on various state and parameter factors. This forms an entropy-regularized reinforcement learning framework tailored for zero-sum stochastic differential games (ERRL-ZSSDGs) operating within a regime-switching jump-diffusion process. The framework leverages the dynamic programming principle to derive Hamilton-Jacobi-Bellman-Isaacs (HJBI) equations, from which equilibrium strategies can be determined. For simpler linear-quadratic problems, semi-analytical solutions are achievable. For more general scenarios, an Actor-Critic policy improvement algorithm is developed to approximate value functions and policies across different regimes. The method's practical application is demonstrated through an investment game, illustrating how temperature parameters and regime transitions influence optimal strategies.