New Framework Enhances Safety and Flexibility in Constrained Reinforcement Learning.

The traditional Constrained MDP (CMDP) framework, while useful for incorporating safety into Reinforcement Learning (RL), lacks support for risk-sensitive constraints. This limitation can lead to suboptimal policies that might mix infrequent catastrophic events with overly conservative behaviors, even when aiming for risk-neutral outcomes. This research addresses this gap by introducing a practical methodology for Utility-Constrained MDPs (UCMDPs). This new framework not only enables the integration of risk-sensitive constraints but also offers significant flexibility by not requiring fixed constraint limits prior to training. Instead, limits can be adjusted within a known range without incurring additional training costs. The proposed agent demonstrates strong empirical performance, consistently matching or surpassing existing baselines across several Safety Gymnasium benchmark tasks. This advancement provides a more robust and adaptable approach to developing safe and effective RL agents, particularly in high-stakes environments.