Adaptive KappaSharp Enhances Preferential Bayesian Optimization

Preferential Bayesian Optimization (PBO) is a technique used to optimize objectives that can only be evaluated through pairwise user comparisons. A common challenge in PBO is that the standard approach, which fits a Gaussian process surrogate, often results in a rank-deficient likelihood Hessian. This deficiency arises because new queries typically form isolated components in the comparison graph, hindering effective model fitting. To overcome this, the researchers propose KappaSharp, a method that applies a diagonal correction to the Hessian during model fitting, specifically targeting areas of high prior uncertainty. An adaptive variant, Adaptive KappaSharp, further refines this by applying corrections only when the surrogate model is confident in recent comparisons. Experiments across 11 benchmarks, including a real-world plasma medicine controller tuning problem, demonstrate that Adaptive KappaSharp significantly outperforms standard PBO, achieving up to a 10.9% improvement.