Contrastive Inverse Dynamics Improves JEPA World Models

Joint-Embedding Predictive Architectures (JEPAs) are designed to learn world models by predicting future embeddings. However, they face a challenge: the objective can lead to a "collapsed" encoder that outputs constant embeddings, rendering the model useless. Existing solutions, like LeWorldModel (LeWM), prevent this by regularizing the latent distribution to match a predefined shape, such as an isotropic Gaussian. This research argues that the anti-collapse pressure can instead originate directly from the transition data. It proposes Action-Contrastive Masked Transition Modeling (AC-MTM), which retains LeWM's forward latent-prediction objective but adds a training-only inverse-dynamics head. This head is trained using Action-NCE, forcing each latent transition to be distinguishable by the action that produced it from other actions in the batch. A collapsed encoder would fail this discrimination task. After training, the inverse branch is discarded, leaving the test-time encoding, forward prediction, and planning identical to LeWM. AC-MTM trains stably from scratch and matches LeWM's performance on standard pixel-control tasks. Crucially, on harder multi-object visual scene tasks, AC-MTM significantly outperforms LeWM (80.0% vs. 58.0% success), suggesting that prescribing a fixed latent geometry can become a bottleneck. This method offers a distribution-free anti-collapse signal without needing target networks, stop-gradients, or pretrained encoders.