Retrieval Grounding Improves Multi-Turn Search Agents

This research addresses a critical limitation in aggregating outputs from multiple large language model (LLM) rollouts, particularly for multi-turn search agents that rely on external document retrieval. Traditional confidence-based voting, which uses internal signals like token log probabilities, performs poorly in this setting due to "copy inflation." This phenomenon occurs when tokens copied from retrieved documents receive artificially inflated log probabilities, flattening confidence scores and weakening the voting mechanism. To overcome this, the authors propose Retrieval-Grounded Voting (RGV). Instead of relying on internal LLM confidence, RGV scores each rollout by measuring the lexical overlap between its final answer and the documents it retrieved. This approach computes the signal outside the "contaminated" context, avoiding the pitfalls of inflated log probabilities and eliminating the need for additional LLM calls for scoring. Evaluations across four search-agent benchmarks and five different LLMs consistently show that RGV outperforms confidence-based voting. It achieves accuracy gains of up to 5.4% and a significant 35% improvement on "minority-correct" questions, where the correct answer is present in only a few rollouts. This demonstrates RGV's effectiveness in robustly aggregating diverse LLM outputs.