ExTS Improves Agentic Search Under Budget Constraints

A new tree-search policy called ExTS (Exploit More, Explore Smarter) has been developed to enhance agentic search performance under strict budget constraints. Traditional Monte Carlo Tree Search (MCTS) often allocates budget inefficiently in scenarios where evaluations are expensive or require multiple model calls, leading to suboptimal exploration and exploitation. ExTS addresses these issues through three core mechanisms: discriminative reward shaping, which helps differentiate candidates even with narrow score distributions; a stochastic virtual child, which estimates the value of new branches based on parent reward history; and quality-conditioned branching, which only expands nodes when their score justifies the budget cost. Across diverse tasks like prompt optimization, code generation, and molecular structure elucidation, ExTS consistently matched or improved upon task-specific tree-search baselines, demonstrating an average relative gain of +5.5% with a single fixed configuration. The research also provides diagnostics to characterize different budget-constrained problems.