LLMs Exhibit Stable Miscalibration, Not Just Fragile Errors

High-confidence errors in large language models (LLMs) are often attributed to fragile internal inference mechanisms, suggesting that these errors are easily perturbed. However, new research introduces the concept of "stable miscalibration," where an LLM confidently provides an incorrect answer that remains robust even when subjected to minor input variations. This implies a deeper, more entrenched form of error. The study employs two key diagnostic tools: an output-level audit score that ranks domains by confidence variation and overconfident mistakes, and an internal sensitivity probe that measures hidden-state movement. Findings indicate that self-critical prompting consistently reduces hidden-state sensitivity across various layers in open-weight models, suggesting a prompt-induced local stabilization. Crucially, the research notes that audit-defined overconfident errors are not necessarily more locally sensitive than confidently correct answers, reinforcing the idea that some high-confidence errors are stable and miscalibrated rather than merely fragile.