Timezone: »

 
Poster
Local Latent Space Bayesian Optimization over Structured Inputs
Natalie Maus · Haydn Jones · Juston Moore · Matt Kusner · John Bradshaw · Jacob Gardner

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #115
in Poster Session 3 »

Bayesian optimization over the latent spaces of deep autoencoder models (DAEs) has recently emerged as a promising new approach for optimizing challenging black-box functions over structured, discrete, hard-to-enumerate search spaces (e.g., molecules). Here the DAE dramatically simplifies the search space by mapping inputs into a continuous latent space where familiar Bayesian optimization tools can be more readily applied. Despite this simplification, the latent space typically remains high-dimensional. Thus, even with a well-suited latent space, these approaches do not necessarily provide a complete solution, but may rather shift the structured optimization problem to a high-dimensional one. In this paper, we propose LOL-BO, which adapts the notion of trust regions explored in recent work on high-dimensional Bayesian optimization to the structured setting. By reformulating the encoder to function as both an encoder for the DAE globally and as a deep kernel for the surrogate model within a trust region, we better align the notion of local optimization in the latent space with local optimization in the input space. LOL-BO achieves as much as 20 times improvement over state-of-the-art latent space Bayesian optimization methods across six real-world benchmarks, demonstrating that improvement in optimization strategies is as important as developing better DAE models.

Keywords: Bayesian optimization · ML for molecules
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Natalie Maus (University of Pennsylvania)
Haydn Jones (Los Alamos National Laboratory)
Juston Moore (Los Alamos National Laboratory)

Juston Moore is a cybersecurity researcher in Los Alamos National Laboratory's Advanced Research for Cyber Systems group. Juston's research bridges statistics, machine learning, and information assurance. His work focuses on large-scale analytics for anomaly detection in unstructured data streams as well as cyber attack attribution.

Matt Kusner (University College London)
John Bradshaw (Massachusetts Institute of Technology)
Jacob Gardner (University of Pennsylvania)

More from the Same Authors