New Kernel Improves Bayesian Optimization Flexibility and Robustness

Bayesian Optimization (BO) is a powerful technique for optimizing expensive black-box functions, but its effectiveness often hinges on selecting a kernel function that accurately models the underlying objective's structure. Traditional kernels can struggle with objectives that exhibit both smooth regions and abrupt changes. This research proposes ALAS (Additive Learnable Alpha-Stable Kernels), a new family of Gaussian Process kernels. ALAS leverages symmetric alpha-stable spectral components, allowing it to learn a "stability parameter" (alpha) directly from the data. This adaptive nature enables the kernel to dynamically adjust its effective smoothness, making it capable of modeling a wider range of objective function behaviors, from very smooth to highly irregular. The paper presents two variants: a single stationary component (ALAS) and a separable version (ALAS-Sep) that learns dimension-wise tail behavior, enhancing robustness for approximately decomposable objectives. Evaluations on standard benchmarks and real-world surrogate functions demonstrate that ALAS consistently delivers strong and robust performance across diverse optimization scenarios.