PathBridger Enhances Offline Reinforcement Learning for Long-Horizon Tasks

Offline goal-conditioned reinforcement learning (GCRL) faces significant challenges when dealing with long-horizon tasks, primarily due to the difficulty of propagating sparse goal-reaching signals over many steps without real-time environment interaction. Existing approaches often rely on improving value estimation or using subgoals to shorten the effective decision horizon. However, many hierarchical methods leave the path between a selected subgoal and the current state implicit, relying on a low-level policy to figure it out. PathBridger addresses this by introducing an explicit mechanism to bridge the gap between subgoal selection and execution. It constructs a direct state-space path towards an intermediate endpoint and then translates this path into a sequence of actions using an inverse dynamics model. This explicit path construction allows for more robust and efficient execution, particularly in complex environments. Experimental results on OGBench tasks, especially multi-object Cube manipulation, demonstrate that PathBridger significantly improves performance. By making the intermediate path explicit, the method enhances the reliability of long-horizon task completion in offline settings.