Frugal Bayesian Optimization Offers Scalable, Efficient Surrogates

This paper introduces "Frugal Bayesian Optimization" (FruBO), a systematic and compute-aware study that re-evaluates the efficiency of Bayesian Optimization (BO) surrogate models. BO is widely used for data-efficient optimization in scientific and engineering fields, but its computational cost is often overlooked. The research benchmarks four surrogate models—Gaussian Processes, Random Forests, NGBoost, and Bayesian Adaptive Spline Surfaces—across eight benchmark functions and nine real-world datasets from diverse domains. The study's key finding is that Gaussian Process-based BO, despite its popularity, consistently incurs the highest time and memory overhead without delivering superior optimization quality or sample efficiency. In contrast, more scalable alternatives, particularly Random Forests, achieve comparable or even better performance while being significantly more computationally frugal. Motivated by these results, the researchers developed a surrogate-recommendation framework within FruBO. This framework predicts the most suitable BO surrogate based on inexpensive dataset characteristics, providing practical guidance for selecting the best model under limited computational and experimental budgets. FruBO establishes a reproducible, compute-aware baseline for BO, enabling more efficient and accessible optimization in various applications.