Probes Detect AI Errors That Confidence Scores Miss

Language models exhibit a "knowing-saying gap," meaning that while internal linear probes can almost perfectly detect corrupted context, this internal awareness doesn't reliably translate into the model's ability to predict its own errors or express accurate confidence levels. For instance, in multi-hop arithmetic, probes detecting corruption don't indicate final answer correctness, and structured confidence formats often collapse to two values with similar error rates. The persistence of probes across reasoning steps also fails to differentiate correct from incorrect outcomes, challenging the hypothesis that persistent probe signals indicate accuracy. This pattern, where models "know" but don't "say" their errors, holds across various model families, including those designed for reasoning. Consequently, real-time probe-based interventions are highly specific to the model and error type. While some interventions like "branch-and-pick" can be net-positive, others like "reprompt" or "replace-prior" risk breaking correct traces as often as they fix incorrect ones. The conclusion is that probe-based monitoring is a necessary complement to verbalized confidence, but effective deployment requires model-aware and error-type-aware routing strategies.