Comparing Uncertainty Quantification for AI Crash Surrogates

Machine learning surrogate models are increasingly being adopted in engineering design to accelerate simulation-driven development, offering rapid predictions that complement time-consuming high-fidelity analyses. However, a significant barrier to their widespread use in safety-critical applications is the lack of reliable uncertainty estimates alongside point predictions. Engineers need to know when a model's prediction might not be trustworthy. This research provides a head-to-head comparison of two popular uncertainty quantification (UQ) methods: Monte Carlo Dropout and Deep Ensembles. These methods were applied to an open-source surrogate pipeline built on NVIDIA PhysicsNeMo, specifically for automotive crash simulation using a steel bumper beam impact problem as a benchmark. A key innovation in this study is the use of concrete dropout, a PhysicsNeMo feature that automatically learns the dropout rate, addressing a common criticism of Monte Carlo Dropout. The study evaluated both methods on identical held-out simulations, comparing their point accuracy, uncertainty band calibration, and computational cost. The findings reveal a crucial trade-off: while deep ensembles are often considered the default gold standard for UQ, this research suggests that well-calibrated, hyperparameter-free uncertainty estimates can be achieved with Monte Carlo Dropout at a significantly lower computational cost. This challenges the prevailing assumption and offers a more efficient alternative for engineering workflows.