RS-Diffuser Enables Risk-Sensitive Offline Reinforcement Learning.

Offline reinforcement learning (RL) is highly appealing for safety-critical applications because it allows policies to be learned from fixed datasets without risky real-world interactions. Diffusion-based methods have recently shown strong performance in offline RL by effectively modeling complex, multimodal trajectory distributions. However, these existing diffusion planners typically operate in a risk-neutral manner, often overlooking rare but potentially catastrophic outcomes that are vital in real-world deployments. To address this, researchers propose RS-Diffuser, a novel risk-sensitive offline diffusion planning framework. RS-Diffuser integrates diffusion-based trajectory generation with distributional value critics. It learns a diffusion planner for future state trajectories, an inverse dynamics model for action decoding, and a Monte Carlo distributional critic that estimates the full return distribution of potential plans using quantile regression. During the sampling phase, RS-Diffuser incorporates a risk-sensitive guidance signal into its denoising process. This signal uses gradients derived from tail-aware objectives, such as Conditional Value at Risk, to direct trajectory generation towards specific risk profiles. A key advantage is that a single trained model can flexibly generate risk-averse, risk-neutral, or risk-seeking behaviors simply by modifying an inference-time risk parameter. Extensive experiments on D4RL benchmarks and risky robot navigation tasks demonstrate that RS-Diffuser achieves state-of-the-art performance, enhancing both overall returns and worst-case robustness while significantly reducing safety violations.