DADiff Improves Reinforcement Learning Policy Adaptation Across Domains.

Transferring policies in reinforcement learning (RL) from a source domain to a target domain is a significant challenge, primarily due to differences in their underlying dynamics. Existing methods often use classifiers or representation learning, but this new research introduces a generative modeling perspective. The proposed framework, DADiff, utilizes a diffusion-based approach to estimate the dynamics mismatch between domains. It does this by analyzing the discrepancy in generative trajectories for predicting the next state. This allows for effective policy adaptation even when only limited interactions with the target domain are available. DADiff offers variants for both reward modification and data selection to facilitate adaptation. Theoretical analysis supports its effectiveness, demonstrating superior performance over existing methods in various environments with different types of shifts.