Self-Diagnosing Models Pinpoint AI Failure Reasons Under Data Shifts

Machine learning models often struggle when deployed in environments different from their training data, a problem known as distribution shift. Current solutions primarily focus on detecting these out-of-distribution samples and quantifying model uncertainty. This new research proposes a more advanced approach: self-diagnosing models that can identify the specific causes of failure under such shifts. These models are designed to learn predictive outputs, uncertainty levels, and a detailed failure attribution signal simultaneously. They use a "failure attribution vector" to categorize unreliability into four distinct types: covariance shift, semantic shift, noise corruption, and adversarial perturbation. This allows for a granular understanding of why a model might be failing, rather than just knowing that it is uncertain. The training incorporates a consistency regularizer to align uncertainty and failure attribution, and new benchmarks were created to evaluate the model's ability to diagnose failure reasons.