Adjustment Speed Proposed as Safety Constraint for Nonstationary RL.

Ensuring safety in reinforcement learning (RL) systems operating in nonstationary environments presents a unique challenge: the system must adapt to environmental changes quickly enough to avoid unsafe transient behaviors. Current safe RL methods typically assume static environments and do not explicitly account for the speed of adaptation as a safety factor. This oversight can lead to dangerous situations when environments evolve over time and the learning system's adaptation lags. This paper proposes a novel safety principle: adjustment speed as a safety constraint for nonstationary RL. The core idea is to define safety based on adaptation feasibility, meaning that future states become unsafe if the required adaptation exceeds the learning system's calibrated recovery capacity. The proposed framework uses learned context representations and short-horizon context forecasts to estimate the demand for adaptation and compare it against the agent's known capacity to recover. When the predicted adaptation demand surpasses the calibrated recovery capacity, the framework proactively intervenes. It tightens the admissible action set and activates an action-level shield to mitigate unsafe behavior before violations occur. Experiments in a nonstationary driving environment demonstrated that this approach significantly reduces safety violations, particularly in short-horizon windows aligned with context changes. Ablation studies further showed that shielding is effective for suppressing peak and tail risks, while optimization-level adjustments help reduce violations conditioned on short-horizon switches.