Selective Verification Optimizes LLM Reasoning for Budget-Aware Deployment

This research explores the optimal allocation of computational resources for test-time reasoning in large language models (LLMs), recognizing that additional reasoning is not always beneficial. Extra processing can either correct errors, be wasted on already correct answers, or even introduce new mistakes. The paper frames this as a deployment allocation problem and introduces SeVRA (Selective Verification for Reasoning Allocation), a serving-layer controller designed to decide whether to retain an LLM's initial answer or trigger an active verification process. Using a frozen Qwen3-4B solver, the researchers logged intervention outcomes and trained recoverability-aware gates based on the visible state of the LLM's initial attempt. On the MathFive benchmark, selective verification achieved 76.3% accuracy, slightly surpassing the 75.5% achieved by always verifying, while critically reducing post-generation tokens by 26.8% and harmful answer flips from 2.2% to 1.0%. However, the study also found that simply increasing the initial solve budget (e.g., an 8,192-token initial solve) could achieve similar accuracy (76.0%) with even fewer total model tokens, suggesting a trade-off. In a transfer learning scenario to GSM8K, the selective policy verified only 3.0% of examples, improving accuracy from 93.4% to 94.5% and cutting verification tokens by 91.2% compared to always verifying. Again, a longer initial solve matched this accuracy with fewer overall tokens. For CommonsenseQA, always-on verification proved detrimental, while Self-Consistency@5 improved accuracy at a five-fold token cost. The key takeaway for deployment is to first optimize the initial reasoning budget, then use selective recovery for scenarios where explicit checks, bounded retries, auditability, or regression-risk control are paramount.