Skip to yearly menu bar Skip to main content


Poster

Maximizing acquisition functions for Bayesian optimization

James Wilson · Frank Hutter · Marc Deisenroth

Room 517 AB #156

Keywords: [ Bandit Algorithms ] [ Gaussian Processes ] [ AutoML ]


Abstract:

Bayesian optimization is a sample-efficient approach to global optimization that relies on theoretically motivated value heuristics (acquisition functions) to guide its search process. Fully maximizing acquisition functions produces the Bayes' decision rule, but this ideal is difficult to achieve since these functions are frequently non-trivial to optimize. This statement is especially true when evaluating queries in parallel, where acquisition functions are routinely non-convex, high-dimensional, and intractable. We first show that acquisition functions estimated via Monte Carlo integration are consistently amenable to gradient-based optimization. Subsequently, we identify a common family of acquisition functions, including EI and UCB, whose characteristics not only facilitate but justify use of greedy approaches for their maximization.

Live content is unavailable. Log in and register to view live content