Transformers Build World Models But Struggle to Maintain Them

Large Reasoning Models (LRMs) have shown impressive capabilities, but their performance on complex planning tasks like the Tower of Hanoi puzzle remains a challenge, particularly for variants beyond the standard formulation. Previous work indicated that these models struggle, and this new research delves into the mechanistic reasons behind these failures.The study first trained smaller Transformers on precomputed solutions for the Tower of Hanoi. Through interpretability techniques, it was discovered that these models develop an "emergent world model"—a geometrically accurate, linearly decodable representation of the puzzle's state space, resembling a Sierpinski triangle. Crucially, this world model was found to be causally involved in the puzzle-solving process.Applying these same techniques to frontier LRMs like Qwen3.6-27B and DeepSeek-R1-Distill-Qwen-32B, the researchers made a surprising observation: these models also encode the Sierpinski world model almost perfectly at the beginning of a prompt. However, they still fail on most tasks with more than three rings. The core issue identified is not the absence of a world model, but its decay during the planning process. Probing at different stages of planning confirmed this degradation, and performance could be partially recovered by re-injecting the initial, well-formed representation. This suggests that the models' failure is one of maintaining necessary representations, rather than an inability to form them.