New Method Improves LLM Agent Performance with Explicit World Models

Large Language Model (LLM) agents often improve their performance by iteratively refining their code or actions based on feedback from an environment. This iterative process typically relies on an implicit understanding of how the environment responds to changes. The new Belief-Calibrated Optimization (BCO) method makes this understanding explicit. BCO involves creating a persistent, in-context document that acts as a "world model." This document records the agent's beliefs about how the environment will react to its edits and is continuously updated as new candidates are evaluated. By making these beliefs explicit and revisable, the agent gains a clearer, more consistent understanding of the environment. Experiments show that BCO leads to higher pass rates on various benchmarks, including memory QA, tool-use, and app agents, compared to control groups without the explicit world model. The benefits persist even when the underlying frozen model is swapped, indicating the scaffold's robustness. An offline analysis confirms that the content of the world model document, not just its form, carries valuable, reusable information for predicting environmental responses.