IADD-TR Boosts Sample Efficiency in Model-Based Reinforcement Learning

Model-based reinforcement learning (MBRL) is a promising approach for achieving sample-efficient decision-making by learning environment dynamics to generate synthetic experience. However, many existing MBRL methods treat the transition model and critic as monolithic predictors, often overlooking the policy-induced data bias. This can lead to entanglement between actions and environmental evolution, and distorted counterfactual value estimates due to uneven action coverage. To address these challenges, researchers propose IADD-TR, a unified framework that combines Intervention-Aware Dynamics Decoupling (IADD) with Targeted Regularization (TR). IADD factorizes environmental transitions into two distinct stages: an action-intervention stage and an action-free natural evolution stage. It uses a zero-action anchor to ensure robust generalization and identifiable latent components. For policy learning, TR is derived from the efficient influence function of a replay-state policy-gradient functional. TR enhances the critic with an action-density-scaled residual correction and optimizes a targeted loss, resulting in doubly robust policy-gradient estimation. Extensive experiments on five MuJoCo tasks demonstrate that IADD-TR achieves competitive returns while significantly improving sample efficiency, making it a valuable advancement in MBRL.