AutoSafe Enables Smooth, Safe Online Reinforcement Learning.

Online reinforcement learning (RL) often faces a dilemma between strictly enforcing safety constraints and maintaining smooth optimization. Traditional methods either use disruptive action interventions for safety or soft constraints that offer limited guarantees. This research introduces AutoSafe, a novel policy architecture designed to bridge this gap. AutoSafe embeds safety monitoring and intervention directly into the action generation process, allowing for a continuous and adaptive shift between optimizing for performance and prioritizing safety. This ensures that the learning process remains smooth and uninterrupted, even as safety measures are actively applied. Empirical results across various benchmarks, including a physical cart-pole system, demonstrate strong safety enforcement without sacrificing learning smoothness.