ReGuide Improves Robot Policies with Self-Correcting Diffusion

Behavior-cloned diffusion policies, while expressive, often struggle with covariate shift, where small deviations from trained states can lead to task failures in robotics. Existing solutions either expand training data, which can be costly, or use test-time guidance, which discards valuable corrective information after execution. This research introduces ReGuide, a novel self-improving framework designed to overcome these limitations. ReGuide leverages Phase-Conditioned Guidance (PCG) to generate corrective rollouts. It identifies drifted-but-recoverable states and applies guidance to steer the policy back towards successful trajectories, ensuring the corrected actions align with the dynamics model's training distribution. Crucially, these successful guided rollouts are not discarded but are absorbed back into the policy. The framework offers two methods for integrating this new data: ReGuide-FT, which fine-tunes the current policy checkpoint, and ReGuide-FS, which retrains from scratch on the augmented dataset. These methods can also be combined and iterated. Experiments on various Robomimic tasks showed ReGuide improving base-policy success rates by 1.3 to 7.7 times, demonstrating its effectiveness in creating more robust and adaptive robotic policies.