Geometric Signatures Predict LLM Reasoning Hardness.

Large Language Models (LLMs) often employ Chain-of-Thought (CoT) reasoning to tackle complex problems by breaking them down into intermediate steps. While much research focuses on the content and length of these reasoning chains, less is understood about their underlying geometric structure within the model's hidden state space. This study delves into the "geometry" of CoT trajectories, treating each reasoning chain as a discrete curve in a high-dimensional space and analyzing it using spectral, positional, and kinematic geometric functions. A key finding is the introduction of the effective dimension, denoted as $d_\rho$, which quantifies trajectory complexity. The research theoretically demonstrates that trajectories exhibiting flatter eigenvalue spectra correspond to harder tasks, implying that the model explores a greater number of hidden dimensions. This suggests a direct link between the geometric complexity of the internal reasoning path and the inherent difficulty of the problem. Furthermore, the study investigates how kinematic features of these trajectories—such as mean position, positional dispersion, initial and current hidden states, mean velocity, mean speed, and speed dispersion—can be used to predict the correctness of a solution even before the generation process is complete. This capability could inform future strategies for early stopping in LLM inference. Experimentally, on mathematical reasoning problems from the MATH500 dataset, $d_\rho$ achieved an impressive 0.93 AUC in distinguishing easy from hard problems. Moreover, kinematic features showed potential to predict correctness using only the first 20% of generated tokens, with these correctness signatures transferring across questions of varying difficulty. This establishes that the shape of an LLM's internal reasoning trajectory offers a principled insight into both task hardness and solution quality.