Learnijoy
ResearchAI ResearchAI Engineering & DevTools

LLMs Overconfident in Causal Judgments, Lack Directness

Amit Kumar, Elnur Adl Zarabi, Suranjana Trivedy, Zhiqian Chen, Lei Zhang, Kaiqun Fu, Taoran Ji· August 26, 2026 View original

Key takeaways

  • LLMs are recall-dominant and overconfident in causal judgments.
  • They struggle to distinguish direct from indirect causal links.
  • Conventional LLM confidence metrics are often unreliable.
  • Agreement across prompts or models offers a more promising, but not definitive, signal.

Who benefits

HealthcareFinanceSocial SciencesAI/ML EngineeringResearch & Development

Summary

A systematic evaluation of 12 LLMs reveals they are recall-dominant and overconfident in causal judgments, frequently misclassifying indirect or reversed non-edges as direct causal links. Conventional confidence metrics are unreliable, while cross-prompt or cross-model agreement offers a more promising, though not statistically significant, signal for trustworthiness.

Large Language Models (LLMs) are increasingly used to provide causal knowledge for structural causal discovery, but their reliability in judging direct causal edges and their confidence levels remain largely unverified. This study systematically evaluated 12 instruction-tuned open-weight models across various causal graphs, prompting strategies, and confidence sources. The evaluation, using a language-only pairwise protocol, yielded three key findings. Firstly, LLM causal judgments are strongly recall-dominant, leading to overly dense graphs with many false-positive edges. Models frequently misclassify indirect or reversed non-edges as direct, and prompting primarily shifts the precision-recall trade-off rather than resolving this overprediction. Model scale improvements diminish on larger graphs and do not eliminate miscalibration. Secondly, LLMs often capture general causal relatedness but struggle with identifying directness or orientation. A significant percentage of indirect and reversed non-edges are misclassified as direct, with high verbalized confidence, indicating substantial overconfidence in structurally incorrect predictions. Thirdly, conventional confidence estimates (like logit-based) are unreliable, often collapsing near 1.0 regardless of correctness. Cross-prompt and cross-model agreement show better mean calibration and discrimination, though their advantages are not yet statistically significant.

Why it matters

Professionals using LLMs for causal inference or knowledge extraction need to be aware of their inherent biases and overconfidence in identifying direct causal relationships, which can lead to flawed decision-making or incorrect scientific conclusions.

How to implement this in your domain

  1. 1Exercise extreme caution when using LLMs to infer direct causal relationships in critical applications.
  2. 2Prioritize human expert validation for any causal graphs generated or suggested by LLMs.
  3. 3Experiment with ensemble methods, like cross-prompt or cross-model agreement, to gauge the reliability of LLM causal judgments.
  4. 4Avoid relying solely on verbalized or logit-based confidence scores from LLMs for causal claims.
  5. 5Develop internal benchmarks to assess LLM performance on causal tasks specific to your domain, focusing on precision and directness.

Original post by Amit Kumar, Elnur Adl Zarabi, Suranjana Trivedy, Zhiqian Chen, Lei Zhang, Kaiqun Fu, Taoran Ji

"arXiv:2608.23660v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to provide prior causal knowledge for structural causal discovery, yet whether their direct-edge judgments and confidence can be trusted remains unclear. We systematically evaluate…"

View on X

Originally posted by Amit Kumar, Elnur Adl Zarabi, Suranjana Trivedy, Zhiqian Chen, Lei Zhang, Kaiqun Fu, Taoran Ji on X · view source

Want to go deeper?

Turn these trends into skills with Learnijoy's hands-on AI & tech courses.

Explore courses

More in AI Research

AI ResearchAI Engineering & DevToolsAI Investing

FraudBench Benchmarks Adversarial Robustness in Financial Risk Assessment

This paper introduces FraudBench, a protocol-sensitive benchmark for evaluating the adversarial robustness of machine learning models in financial fraud and credit-risk detection. It demonstrates that robustness conclusions are highly dependent on how domain-specific constraints and attacker capabilities are incorporated into the evaluation protocol.

Xitong Zeng, Zhaoge Bi, Yitian Yang, Huaming Chen, Quan Z. ShengAug 26, 2026
AI ResearchAI Engineering & DevTools

Persistent Cross Entropy Extends Topological Data Analysis

This paper introduces Persistent Cross Entropy (PCE), a novel extension of cross-entropy to persistence diagrams, which are used in topological data analysis. PCE bridges different event spaces of diagrams using an induced probability, enabling new applications like distinguishing diagrams with similar persistent entropy and separating causal directions in dynamical systems.

Sijin Yeom, Jae-Hun JungAug 26, 2026
AI ResearchAI Engineering & DevTools

Bridging Numerical PDE Solvers and Neural Emulators for Faster Simulation

This thesis explores the deep connections between traditional numerical solvers for Partial Differential Equations (PDEs) and neural emulators, arguing that they are more alike than different. It proposes that insights can flow profitably in both directions, leading to faster and more efficient scientific and engineering simulations.

Felix KoehlerAug 26, 2026