Efficient Bayesian Deep Ensembles Offer Calibrated Uncertainty

Deep ensembles are known for their strong predictive performance and robust uncertainty estimates, but they can be computationally expensive. This research proposes an efficient Bayesian deep ensemble method specifically designed for predictive regression, aiming to improve interpretability while maintaining competitive performance and computational efficiency. The method is built on three core design principles to provide calibrated uncertainty estimates suitable for both standalone predictions and integration into larger learning systems. First, the approach uses a low-dimensional ensemble representation, where predictions are expressed as a combination of a small number of pre-trained neural predictors. This design ensures scalable inference, with computational costs dependent on the ensemble size rather than the entire dataset size. Second, it employs closed-form Bayesian aggregation to combine ensemble predictions. This yields interpretable posterior weights and calibrated uncertainty without the need for complex approximate inference techniques. Finally, the method relies on independent training of multiple neural networks. This strategy produces diverse predictive representations within the ensemble, which in turn enhances robustness and improves the calibration of uncertainty estimates. Empirical evaluations on standard regression benchmarks confirm that this proposed approach achieves competitive predictive performance while delivering reliable uncertainty estimates across various settings, making it a valuable tool for applications requiring both accuracy and confidence quantification.