StepGuard Improves Web Navigation Agents with Single-Step Calibration.

Web navigation agents, which interact with web pages to achieve natural language goals and provide answers, often suffer from fragility at individual steps. This fragility stems from misaligned rewards and the propagation of errors throughout the navigation process. Current methods, despite leveraging vision-language models and reinforcement learning, have not fully resolved these issues. StepGuard introduces a novel framework to address these challenges through single-step calibration. It incorporates Dynamic Dual-Policy Optimization (DDPO), a mechanism that dynamically switches between a navigation-first mode for exploration and an answer-first mode for question-answering. This dynamic switching effectively mitigates conflicts between different reward signals. Furthermore, StepGuard proposes Confidence-Guided Adaptive Navigation Reflection (CANR) to calibrate single-step errors. CANR estimates per-step confidence, triggering reflection only when necessary. It then uses contrastive rewards to encourage the agent to self-correct inaccuracies. Together, DDPO and CANR form the core of StepGuard, which has demonstrated significant improvements in both navigation and answer accuracy, setting new state-of-the-art performance on standard web navigation benchmarks.