Heckman Correction Improves AI Uncertainty Estimates with Selection Bias

Machine learning models often encounter training data that is subject to selection bias, meaning observations are only available under certain conditions (e.g., loans granted, tests ordered). Standard methods like importance weighting or covariate-shift correction assume this selection is ignorable given observable features. However, this paper revives the Heckman correction from econometrics to tackle the more complex scenario where selection depends on unobservable factors. The proposed method integrates a deep outcome network with a linear selection head and a joint bivariate-normal likelihood, ensembled for predictive variance. Experiments with a controlled data generator show that traditional deep ensembles and importance weighting fail to correct for overconfidence in regions where data was avoided due to unobservable correlations. The Heckman correction, when an appropriate instrument variable is present, significantly restores coverage of uncertainty intervals, demonstrating its ability to correct for conditional bias. This approach provides better-calibrated uncertainty estimates, especially in selected-against regions, offering a more robust solution for real-world data challenges.