Survey Reveals LLM Numeracy Limitations, Proposes Improvement Paths

Despite their impressive performance on complex mathematical reasoning benchmarks, Large Language Models (LLMs) often struggle with basic numerical tasks like magnitude comparison, arithmetic, and handling fractions. This survey delves into these fundamental limitations, proposing the Numerical Grounding Framework (NGF) to better understand LLM numeracy. The NGF breaks down numeracy into two core components: Representational Grounding, which involves mapping numerical forms to their values and equivalent representations, and Procedural Grounding, which concerns executing arithmetic operations correctly. The paper reviews existing diagnostic benchmarks, common failure modes, and structural explanations related to tokenization, positional encoding, and data distribution. It also evaluates frontier models across various numerical tasks, comparing different intervention strategies. While architectural changes can help models trained from scratch, practical solutions for users of pre-trained models include fine-tuning, reasoning scaffolds, and external tools.