Timezone: »

 
PI is back! Switching Acquisition Functions in Bayesian Optimization
Carolin Benjamins · Elena Raponi · Anja Jankovic · Koen van der Blom · Maria Laura Santoni · Marius Lindauer · Carola Doerr

@
in Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems »

Bayesian Optimization (BO) is a powerful, sample-efficient technique to optimize expensive-to-evaluate functions. Each of the BO components, such as the surrogate model, the acquisition function (AF), or the initial design, is subject to a wide range of design choices.Selecting the right components for a given optimization task is a challenging task, which can have significant impact on the quality of the obtained results. In this work, we initiate the analysis of which AF to favor for which optimization scenarios. To this end, we benchmark SMAC3 using Expected Improvement (EI) and Probability of Improvement (PI) as acquisition functions on the 24 BBOB functions of the COCO environment. We compare their results with those of dynamic schedules which aim to use EI's explorative behavior in the early optimization steps, and then switch to PI for a better exploitation in the final steps. We also compare this to a random schedule and round-robin selection. We observe that dynamic schedules oftentimes outperform any single static one. Our results suggest that a schedule that allocates the first 25% of the optimization budget to EI and the last 75% to PI is a reliable default. However, we also observe considerable performance differences for the 24 functions, suggesting that a per-instance allocation, possibly learned on the fly, could offer significant improvement over the state-of-the-art BO designs.

Author Information

Carolin Benjamins (Leibniz University Hanover)
Elena Raponi (Technical University of Munich, Sorbonne Université)

Elena Raponi is a Postdoctoral Researcher at the Technical University of Munich, Chair of Computational Mechanics, and Sorbonne University, LIP6 Department. Her position is funded by a DAAD Prime Postdoctoral Fellowship, which was awarded in December 2021. Previously, she was a postdoctoral fellow in the Natural Computing Research Group at the Leiden Institute of Advanced Computer Science. She received her Ph.D. in Applied Mathematics from the University of Camerino, Italy, in May 2021. Her research mainly focuses on surrogate-based optimization in continuous domains, high-dimensional Bayesian optimization, and analytical and numerical modeling techniques for the optimization of geometries and materials in structural mechanics. She has published 1 book chapter, 5 conference papers, and 9 journal articles in reputed journals such as Computer Methods in Applied Mechanics and Engineering, Composite Structures, and Reports on Progress in Physics. She is a co-developer of the open-source package [GSAreport](https://github.com/Basvanstein/GSAreport) and contributed to the development of the library [Bayesian-Optimization](https://github.com/wangronin/Bayesian-Optimization).

Anja Jankovic (Sorbonne University)
Koen van der Blom (Sorbonne University)
Koen van der Blom

Koen van der Blom is a post-doctoral researcher at the LIP6 of Sorbonne Universite in Paris, France. He received his Ph.D. in 2019 on multi-objective evolutionary optimisation for early-stage building design from Leiden University (the Netherlands), and his thesis received an honourable mention for the 2020 ACM SIGEVO Best Dissertation Award. His research interests include the accessibility of meta-algorithms such as automated algorithm selection and configuration, evolutionary computation for real-world applications, and multi-objective optimisation.

Maria Laura Santoni (University of Camerino)
Marius Lindauer (Leibniz Universität Hannover)
Carola Doerr (CNRS & Sorbonne University)

More from the Same Authors