Dueling World Models Improve Action-Specific Predictions

This research addresses a critical limitation in latent world models: their tendency to become "action-blind" when scenes contain motion not controlled by the agent. In such scenarios, predictions for different actions become indistinguishable, even as training loss improves. The proposed solution, "Dueling World Models," leverages an advantage-style decomposition, similar to that used in value functions, to create clean, controllable action channels. By subtracting the mean effect of a prediction over all actions, the method effectively cancels out action-independent variations, which are typically where distractors reside. This innovative approach requires no additional machinery like reconstruction, task reward, or auxiliary objectives. It can be applied post hoc to any action-conditioned world model, including frozen pretrained ones, to surface the agent's own effect. The method has been validated across various environments, including gridworlds, synthetic generators, continuous control tasks, and Atari games, demonstrating its ability to recover action-specific effects where entangled predictors fail.