SearchAuditor Diagnoses Failures in Long-Horizon Search Agents

Deep search agents, which interact with the web over long horizons to answer complex questions, are prone to subtle reasoning errors that can propagate and lead to incorrect answers. Diagnosing these failures is challenging due to the length and complexity of execution traces, often exceeding human capacity for manual inspection. To address this, a new benchmark, SearchAuditBench, and a framework, SearchAuditor, have been developed. SearchAuditBench comprises 1,243 failed trajectories from various open-weight models across five deep-search benchmarks, each expertly annotated with error steps, root causes, and repair suggestions. Building on this, SearchAuditor is a multi-perspective auditing framework that effectively localizes, attributes, and repairs these search-agent failures through evidence-grounded adjudication. Experimental results show that even advanced baseline models, when powered by frontier LLMs like GPT-5.5, achieve only a 26.6% end-to-end pass rate. In contrast, SearchAuditor consistently outperforms all baselines, achieving a 32.3% end-to-end pass rate. Furthermore, its repairs enable agents to better recover from errors, highlighting its significant potential for improving the reliability of long-horizon search agents.