Model Calibration Fails with Human Label Ambiguity

Temperature scaling is a widely used post-hoc method to calibrate the confidence of deep learning models. However, its theoretical foundation assumes that ground-truth labels are deterministic and one-hot. This study investigates what happens when this assumption is violated, specifically when labels are soft, crowd-sourced, or genuinely distributional, reflecting real human disagreement rather than simple annotation noise. Using datasets with human-annotated soft label distributions (CIFAR-10H and ChaosNLI), the researchers evaluated models of varying scales across vision and language modalities. They consistently found a "soft-label calibration gap": temperature scaling calibrated on hard, one-hot labels performed worse than an oracle calibrated directly on soft labels. This gap was observed in all nine configurations tested. The study also found that this calibration gap generally increases with model scale in the vision domain and for certain language tasks, and is substantially larger in the language domain compared to vision. These findings suggest that current calibration protocols, which often rely on majority-vote labels, systematically misrepresent model reliability when label ambiguity is inherent, posing direct consequences for deploying AI in sensitive or safety-critical environments.