New Method Improves LLM World Model Representation Quality and Performance

Current large language model (LLM) architectures used in world models often struggle with accurately representing their understanding of an environment, even if their predictions seem good. This discrepancy arises because these models can bypass their internal "latent state" when making predictions, leading to a lack of identifiability in what they truly understand. This paper addresses this by re-introducing the classical principle of strict latent state mediation, ensuring that all predictions must flow through the model's internal representation. The researchers developed a new framework for text-based environments, which are challenging due to the discrete and non-differentiable nature of textual latent states. They propose using interpretable, variable-length textual latent states and a reinforcement learning technique called factorized GRPO (fGRPO). This method enforces strict mediation during training, preventing the model from ignoring its internal bottleneck. Experiments on TextWorld and ScienceWorld environments demonstrated substantial improvements. While one-step prediction accuracy was maintained, the method led to up to 57% gains in representation quality and 98% improvements in long-term rollout performance, especially noticeable in more complex and longer-horizon tasks. This indicates a more robust and genuinely informed internal model of the world.