LLM Confidence Reflects Latent Decision Variables

Dharshan Kumaran, Viorica Patraucean, Maks Ovsanikov, Petar Veli\v{c}kovi\'c, Nathaniel Daw· July 15, 2026 View original

Summary

Research shows that answer logits in multimodal language models often behave as monotonic readouts of a latent decision variable, aligning with statistical decision confidence (SDC) theory. This provides a computational account of confidence, though complex reasoning tasks still pose boundaries.

For large language models (LLMs) to be trustworthy, their confidence in an answer must reliably reflect its correctness. While existing work often evaluates confidence by its predictive power or calibration, a deeper question remains: what does the confidence signal itself truly represent? Is it a normative reflection of a latent decision variable, or merely a heuristic preference?This study addresses this by applying Statistical Decision Confidence (SDC), a framework from computational neuroscience. By treating the answer-logit difference (LD) as a potential readout of a latent decision variable, researchers tested SDC's qualitative signatures. Across several perceptual discrimination and memory tasks, involving both multimodal non-reasoning and reasoning models, LD consistently satisfied these signatures. This indicates that, in these contexts, answer logits function as monotonic readouts of a latent decision variable, rather than simple heuristic scores.However, in more complex visual reasoning tasks, while LD still predicted correctness, the full geometric signatures of SDC were absent. This highlights the current limitations of the framework when explicit normative process models are unavailable. The findings offer a computational explanation for confidence in multimodal language models and establish SDC as a unifying framework for studying confidence across biological and artificial intelligence.

Why it matters

Understanding the computational basis of LLM confidence is crucial for building more reliable and interpretable AI systems, enabling professionals to better trust and utilize AI outputs in critical decision-making scenarios.

How to implement this in your domain

  1. 1Integrate confidence scores derived from answer logits into AI system interfaces to provide users with transparency.
  2. 2Develop strategies to improve model calibration based on the understanding that logits reflect a decision variable.
  3. 3Use confidence signals to filter low-confidence outputs or flag tasks requiring human oversight.
  4. 4Apply the SDC framework to evaluate and enhance the trustworthiness of AI models in new applications.

Who benefits

AI/ML DevelopmentHealthcareFinanceLegalCustomer Service

Key takeaways

  • LLM answer logits often reflect a latent decision variable, not just heuristic preference.
  • Statistical Decision Confidence (SDC) provides a framework for understanding this.
  • This holds true for perceptual and memory tasks in multimodal models.
  • Complex reasoning tasks still present challenges for full SDC application.

Original post by Dharshan Kumaran, Viorica Patraucean, Maks Ovsanikov, Petar Veli\v{c}kovi\'c, Nathaniel Daw

"arXiv:2607.12447v1 Announce Type: new Abstract: Reliable confidence -- the probability that a model's own answer is correct -- is essential for the trustworthy deployment of language models. Existing work has largely evaluated confidence by how well it predicts correctness and wh…"

View on X

Originally posted by Dharshan Kumaran, Viorica Patraucean, Maks Ovsanikov, Petar Veli\v{c}kovi\'c, Nathaniel Daw on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses