New Estimators Trace Uncertainty Sources in Deep Learning

Understanding and quantifying uncertainty in deep learning predictions is crucial for deploying these models reliably in real-world applications. This new research focuses on distinguishing between two primary sources of uncertainty: aleatoric uncertainty, which stems from inherent noise or variability within the data itself, and epistemic uncertainty, which arises from the model's lack of knowledge due to scarce data in certain regions of the input space. The study introduces an adaptation of two classical statistical estimators to the context of modern deep learning. This approach utilizes recent advancements in approximate Fisher Information Matrices, enabling it to scale effectively to complex, contemporary neural network architectures. By doing so, it provides a more granular understanding of how each individual test point is affected by these distinct sources of uncertainty. Experimental results demonstrate the practical utility of these new estimators. They offer clearer insights into which type of uncertainty dominates for specific predictions, thereby enhancing the robustness and trustworthiness of deep learning models in critical applications where understanding prediction confidence is paramount.