Timezone: »

 
Bayesian optimization by density ratio estimation
Louis Tiao · Aaron Klein · Cedric Archambeau · Edwin Bonilla · Matthias W Seeger · Fabio Ramos

Fri Dec 11 10:45 AM -- 11:00 AM (PST) @
in Meta-Learning »

Bayesian optimization (BO) is among the most effective and widely-used blackbox optimization methods. BO proposes solutions according to an explore-exploit trade-off criterion encoded in an acquisition function, many of which are derived from the posterior predictive of a probabilistic surrogate model. Prevalent among these is the expected improvement (EI). Naturally, the need to ensure analytical tractability in the model poses limitations that can ultimately hinder the efficiency and applicability of BO. In this paper, we cast the computation of EI as a binary classification problem, building on the well-known link between class probability estimation (CPE) and density ratio estimation (DRE), and the lesser-known link between density ratios and EI. By circumventing the tractability constraints imposed on the model, this reformulation provides several natural advantages, not least in scalability, increased flexibility, and greater representational capacity.

Author Information

Louis Tiao (University of Sydney)
Aaron Klein (AWS Berlin)
Cedric Archambeau (Amazon)
Edwin Bonilla (CSIRO's Data61)
Matthias W Seeger (Amazon Development Center)
Fabio Ramos (University of Sydney, NVIDIA)

More from the Same Authors