ReGRPO Enhances Tool-Using AI Agents with Reflection.

Tool-augmented vision-language models (VLMs) are designed to solve complex, multi-step tasks by integrating external tools, but they often exhibit fragility, especially when encountering tool failures. Current training methods, such as supervised fine-tuning (SFT), primarily rely on successful trajectories, providing insufficient guidance for recovery from errors. Similarly, sparse reinforcement learning rewards at the trajectory level offer limited insight into specific failure points or how to rectify them. To overcome these limitations, researchers have introduced ReGRPO (Reflection-augmented Group Relative Policy Optimization). This framework enables tool-using agents to learn from their mistakes through a structured, reflection-guided correction mechanism. At its core is a reflective data engine that deliberately executes near-miss actions to gather grounded failure observations. These observations are then used to construct "Reflection-of-Thought" triplets, comprising the error type, evidence, and a fix plan, which are paired with corrected actions for initial SFT. Following this warm-start, ReGRPO jointly optimizes reflection tokens and corrective actions within local trajectories using group-relative advantages. The framework also incorporates a reflection-cost term to prevent unnecessary reflection, ensuring efficiency. Experiments on benchmarks like GTA and GAIA demonstrate that ReGRPO consistently surpasses strong open-source baselines, achieving superior results among comparable open-source controllers by significantly improving an agent's ability to recover from tool failures.