New Thermodynamic Signatures Detect LLM Hallucinations

This research introduces a new method for detecting hallucinations in large language models (LLMs) by analyzing the "thermodynamic signatures" of their reasoning processes. The proposed technique, called Free-Energy Signatures (Fes), leverages the spectrum of attention-derived graph Laplacians within each LLM layer. Fes goes beyond simple eigenvalue summaries by extracting a richer set of spectral descriptors, including partition function, free energy, spectral entropy, heat capacity, and the random-matrix-theory (RMT) spectral form factor. The authors prove the stability and expressiveness of Fes and provide a PAC bound on its detection performance. Empirical evaluations across six open-weight LLMs and six benchmarks demonstrate that a lightweight probe using Fes descriptors achieves the highest aggregate AUROC for hallucination detection among attention-spectral baselines, significantly outperforming previous methods. Additionally, an unsupervised RMT-deviation score provides a label-free detection option, revealing that correct generations exhibit Wigner-Dyson-like spectral statistics, while hallucinations show Poisson-like statistics.