Delta-JEPA Improves World Models with Action-Sensitive Latent Dynamics.

Learning effective visual world models for AI planning requires latent dynamics that are compact yet highly sensitive to actions. Traditional reconstruction-free joint-embedding objectives often struggle with this, sometimes collapsing into representations that are insensitive to the actions taken. This research introduces Delta-JEPA, an innovative end-to-end world model designed to overcome this limitation. Delta-JEPA augments latent forward prediction with a novel Latent Difference Action Decoder (LDAD). Instead of inferring actions from concatenated endpoint embeddings, LDAD reconstructs the executed action directly from the latent displacement between consecutive observations. This displacement-level supervision effectively regularizes the transition geometry, preventing latent collapse and ensuring that different actions induce distinguishable latent changes, which is crucial for rollout-based planning. The model avoids pixel reconstruction and distribution-matching regularizers, relying solely on latent prediction and action reconstruction. Experiments across various visual continuous-control tasks show that Delta-JEPA significantly improves planning performance compared to existing JEPA-based and representation-learning baselines, demonstrating the effectiveness of supervising latent differences for action-sensitive world model learning.