LLMs Learn Adaptive Reasoning to Optimize Compute Allocation

Large language models (LLMs) typically operate with a fixed computational budget for reasoning, leading to inefficiencies where easy problems are over-computed and difficult ones are under-computed. A new research paper introduces an adaptive reasoning approach that allows an LLM to dynamically decide how much "thought" to apply to each problem. The model learns to choose one of three reasoning modes – "NoThink" for quick answers, "Short" for brief reasoning, or "Long" for extended reasoning – as the first token of its response. This choice is learned through Group Relative Policy Optimization (GRPO) with a shaped reward system and distinct token caps for each mode. Evaluated on the MATH benchmark, a 1.5B distilled model trained with this method maintained accuracy close to the baseline while reducing the average response length by 41%. Interestingly, the model effectively sorts problems by difficulty, with simpler modes sometimes outperforming "Long" on easier tasks. The adaptive reasoning also transferred well to other benchmarks like GSM8K, yielding significant token reductions and even higher accuracy in some cases.