Reinforcement Learning for Continuous-Time Jump Markov Decision Processes.

Reinforcement Learning (RL) in Continuous-Time Jump Markov Decision Processes (CTJMDPs) presents a significant challenge, particularly when dealing with general discrete state spaces and mixed continuous/discrete action spaces. Existing continuous-time RL techniques often rely on continuous state spaces and advanced semimartingale theory, which are not directly applicable to CTJMDPs that lack the algebraic structures of Euclidean spaces. This paper aims to bridge this theoretical and practical gap. The researchers establish the foundational theory for Q-learning within CTJMDPs and introduce novel model-free Q-learning algorithms. To manage the exploration-exploitation trade-off, they formulate an entropy-regularized continuous-time control problem that incorporates stochastic policies. This approach provides a more direct and conceptually sound method compared to simply discretizing time and approximating CTJMDPs with discrete-time Markov Decision Processes. Numerical experiments, specifically in the domain of network dynamic pricing, demonstrate the effectiveness of the proposed RL algorithm. It consistently learns policies that are near-optimal and significantly outperforms standard benchmark methods. This indicates superior solution quality and effective scalability, making it a powerful tool for complex operational problems.