New World Models Improve Web Agent Action Selection

Recent advancements in web agents often rely on world models to simulate future web states and select optimal actions. These models typically predict fixed representations like HTML, but this objective doesn't align well with the downstream ranker's need to differentiate between various candidate actions. This research proposes a novel training objective called "predicted-state matching." This new objective ensures that the predicted web state representation can effectively distinguish the actual outcome from states resulting from alternative actions. The models were trained using a specialized branching web-agent dataset, derived from WebArena Go-Browse trajectories, which includes multiple alternative actions and their resulting states for each decision point. Experiments demonstrated that this approach significantly outperforms traditional supervised next-state prediction, leading to improved action ranking and higher end-to-end task success rates on benchmarks like WebPRMBench and WebArena-Lite.