MetaResearcher: Scaling Deep Research Agents with Self-Reflective Reinforcement Learning

Current deep research agents, while capable of information gathering, face limitations in training due to static environments, simple fact-retrieval tasks, and inefficient outcome-based reinforcement learning. A new framework, MetaResearcher, aims to overcome these challenges by scaling agent training across four key dimensions. First, it introduces an "Evolving Virtual World" that incorporates temporal dynamics and adversarial misinformation. This forces agents to develop skills in assessing source credibility and resolving conflicting information over time. Second, the framework designs "Discovery-Oriented Tasks," such as hypothesis generation and contradiction resolution, moving beyond basic fact retrieval to encourage more genuine research behaviors. Third, MetaResearcher employs a "Self-Reflective Meta-Reward" mechanism within the GRPO framework. This mechanism optimizes for answer correctness, search path efficiency, reflection depth, and tool call diversity, directly addressing the common issue of repetitive action loops in previous agent designs. Finally, a "Heterogeneous Multi-Agent Swarm" architecture, comprising specialized Scout, Filter, and Synthesizer models, enables agents to learn collaborative research strategies through coordinated reinforcement learning. Built on the LiteResearcher infrastructure, this framework promises significant improvements in benchmark performance and robustness under adversarial conditions, all while maintaining zero marginal API cost for training.