Agents Improve World Models by Budgeted Environment Probing.

Long-horizon language agents operate with an internal model of the world that evolves as they make decisions. If this internal model drifts from reality, it can lead to failures later in a task, even before the failing action is taken. This research explores a direct mechanism to address this: allowing an agent to "ask the environment" about a specific belief field and update its world model before committing to a task action. This approach treats environment interaction as a scarce calibration resource, not just a means to advance the task. The proposed method introduces a budgeted probing operator for structured belief tables. The study highlights that the utility of probes varies; procedural beliefs, like tool dependencies, can often be fixed with targeted checks, but these checks consume valuable steps. Spatial beliefs, such as object locations, are more reliant on structural cues, and an agent's self-confidence can be misleading when the environment changes unobserved. A type-stratified analysis formalizes this trade-off between probing and action. Controlled experiments demonstrate that incorporating mid-planning environment evidence significantly reduces terminal world-model error, particularly when the probing policy aligns with the task's structure. This suggests a more efficient way for agents to maintain accurate internal representations of their environment.