Dreamer-SAC Boosts Sample Efficiency in Autonomous Driving RL

Reinforcement learning (RL) for autonomous driving often faces a critical trade-off: achieving high data efficiency typically comes at the cost of increased model bias. While world models can reduce the need for extensive real-world interactions, optimizing policies based on these learned dynamics remains sensitive to prediction errors. Researchers introduce Dreamer-SAC, a novel framework designed to address these challenges. It combines a recurrent state-space world model with an off-policy soft actor-critic algorithm, which is trained directly within the latent space. The framework utilizes a blend of real environment interactions and short-horizon generated trajectories, enhanced by n-step target estimation and multi-objective supervision. Evaluations in autonomous driving scenarios, focusing on both efficiency and safety, show that Dreamer-SAC consistently outperforms leading RL baselines like DreamerV3, SAC, and PPO. It achieves superior performance with substantially fewer real environment interactions. The study also highlights that short-horizon latent rollouts offer the best balance between additional training signals and accumulated model bias, and n-step target estimation is more effective than one-step temporal-difference targets for leveraging predicted experience.