LLMs Show Sensitivity to Problem Representation in Math Solving.

New research explores how Large Language Models (LLMs) perform when solving mathematical problems presented in different, yet mathematically equivalent, formats. The study systematically varied surface representations, including story problems, word-equations, symbolic equations, and paraphrases, to test five contemporary LLMs. A key finding is substantial representational sensitivity, meaning models frequently change their correctness across these equivalent formulations, with noticeable "flip rates" between different problem types. Even subtle changes in phrasing through isomorphic reformulations were observed to degrade performance despite the mathematical structure remaining identical. The researchers also evaluated a condition where LLMs externalized their reasoning as executable Python code, which was then run for validation. While this "code-augmented" approach revealed strong latent reasoning capabilities in some models that struggled with direct prompting, it did not uniformly improve robustness. Instead, failure modes shifted from opaque reasoning errors to issues like protocol violations or execution failures. The study concludes that reasoning scaffolds do not eliminate representational brittleness but introduce new trade-offs in correctness, reliability, latency, and cost. It emphasizes that problem representation should be a primary interface design variable in LLM evaluation and deployment, especially for AI-assisted problem-solving systems.