New Framework Quantifies Uncertainty Using Highest Density Regions

Uncertainty quantification (UQ) is a critical component for ensuring reliable decision-making, particularly in safety-critical applications of probabilistic machine learning. For regression tasks, current dominant scalar UQ methods often rely on pointwise predictive risk, which can yield counterintuitive results when the target statistic deviates from the conditional expectation. A new framework, QUEST (Quantifying Uncertainty via highest dEnSiTy regions), is introduced as an alternative approach to UQ. QUEST characterizes uncertainty by measuring the volume of the most probable subset within a distribution's support, evaluated at various robustness parameters. This method establishes connections with classical statistics from information theory and economics. Crucially, QUEST measures of both epistemic (model) and aleatoric (data) uncertainty are shown to satisfy a set of axioms adapted from the UQ literature, including monotonicity under distributional spread and invariance to location shifts, which popular alternatives often fail to meet. Selective prediction benchmarks confirm that QUEST performs favorably against standard measures like variance and differential entropy, offering a more robust and intuitive way to quantify uncertainty.