New Framework Challenges LLM Confidence Evaluation Metrics

A new study critically examines the prevailing methods for evaluating the confidence of Large Language Models (LLMs), arguing that traditional calibration metrics are inadequate. The researchers contend that calibration alone fails to ensure that LLM confidence estimates represent coherent probabilistic beliefs, often allowing for inconsistent or illogical estimations. They highlight that models can assign lower confidence to logically simpler questions, and interventions aimed at improving calibration do not necessarily fix these underlying structural issues. To address these shortcomings, the paper introduces a novel framework called C1 metrics, which evaluates LLM confidence along three crucial dimensions: structural coherence, faithfulness, and usefulness. Their findings indicate that despite appearing well-calibrated, current LLM confidence estimates frequently violate these fundamental conditions for coherent probabilities. This work provides a new lens for measuring and ultimately closing the gap between reported confidence and true probabilistic validity in LLMs.