New Method Uncovers Hidden Math-Reasoning Capabilities in LLMs

Current methods for evaluating the mathematical reasoning abilities of large language models, particularly the common "pass@k" metric which relies on multiple sampling attempts, may be underestimating their true capabilities. Researchers have identified a significant "blind spot" where problems are incorrectly classified as too difficult for the model to solve. A new diagnostic approach, which combines greedy decoding with targeted internal model perturbations via activation grafting, demonstrated that a substantial percentage of these previously "unsolvable" math problems could actually be solved. This suggests that the models possess the underlying knowledge to tackle these challenges, but standard sampling techniques fail to access it. The study highlights that these difficult problems are structurally identifiable within the model's internal representations, indicating that improved inference strategies could unlock latent reasoning potential.