GRAPE Boosts High-Dimensional Bayesian Optimization Efficiency

Researchers have introduced GRAPE, a novel two-stage framework designed to enhance the query efficiency of high-dimensional Bayesian optimization, a critical challenge in machine learning and scientific discovery. Traditional local Bayesian optimization methods often prioritize merely descending over making substantial progress, resulting in many small, inefficient steps. GRAPE addresses this by first employing a gradient refinement stage, which uses a closed-form acquisition function to sharpen the local gradient posterior. Following this refinement, the second stage, "Progress-Aware Exploitation," selects update directions that maximize the expected decrease, conditional on a descent. Theoretical analysis confirms that the gradient refinement monotonically reduces local uncertainty and that the progress-aware direction converges to true steepest descent as the posterior becomes sharper. Empirical evaluations demonstrate GRAPE's superior query efficiency across various high-dimensional tasks, including black-box adversarial attacks where it achieved a 5.4x speedup, and large language model prompt optimization, where it reduced final average regret by 3.8 log-units compared to the next best method.