WorldCycle Improves Long-Horizon Video World Models with Self-Verification

Interactive video world models are crucial for AI agents to perform long-horizon planning and exploration, but they often suffer from compounding errors over extended sequences of actions. A major challenge in improving these models with reinforcement learning (RL) is the "verification bottleneck," where there's no ground truth for future states to measure long-term drift after arbitrary action sequences. A new framework, WorldCycle, tackles this by leveraging the insight that reversible action cycles can enable self-verification. By constructing action sequences that, when combined with their inverse, should return to the initial state, WorldCycle generates annotation-free supervision for long-horizon correctness. WorldCycle optimizes two rewards: a spatial closure reward ensuring symmetry between mirrored forward and reverse segments, and a temporal consistency reward aligning states across repeated cycle executions. This approach forces the model to learn actions as consistent state operators, not just memorized patterns. The framework significantly reduces state returning drift by up to 44% and improves composite-action accuracy nearly fourfold, providing a vital foundation for physically grounded world models.