Physics-Informed RL Enhances Safety in Industrial Control Systems

Deep Reinforcement Learning (DRL) offers powerful control capabilities for complex industrial cyber-physical systems (ICPSs). However, its exploratory nature often carries the risk of violating critical hardware safety limits, which are typically managed through intricate reward shaping. This approach can be cumbersome and prone to errors. This work-in-progress introduces a novel method to enhance safety in DRL. It integrates a differentiable physics model directly into the actor's loss function within a Proximal Policy Optimization (PPO) framework. By simulating short-horizon future trajectories during the training phase, the policy is proactively penalized for any predicted safety violations. Evaluated on a simulated 1-degree-of-freedom helicopter testbed with strict pitch constraints, this physics-informed regularization significantly reduced constraint violations. Crucially, it achieved this safety improvement without compromising the system's ability to reliably track its target, offering a more robust and direct way to enforce safety in DRL-controlled ICPSs.