LLMs Improve Reasoning with Test-Time Self-Correction and Refinement.

Large Language Models (LLMs) often struggle with complex reasoning tasks, and current methods like wider sampling or verifier-based selection have limitations. Wider sampling can lead to repetitive answers, while verifier performance depends on external reward models. Researchers have introduced a novel verifier-free breadth-depth refinement framework to address these issues. This new approach involves sampling multiple independent reasoning paths, then refining each path through iterative self-critique and self-correction. Finally, the refined answers are aggregated using majority voting. This dual strategy allows for diverse initial attempts (breadth) while simultaneously fixing local reasoning errors (depth) before a final decision is made. The method has shown significant improvements over existing techniques, including greedy decoding, majority voting, and verifier-based best-of-N, across several challenging benchmarks like MATH500 and AMC. For instance, it boosted Qwen2.5-1.5B's accuracy on MATH500 to 58.0% and on AMC to 32.5%, demonstrating that test-time compute is more effective when used for refining trajectories rather than just increasing sample size or relying on external verifiers.