Timezone: »

 
Towards Automated Design of Bayesian Optimization via Exploratory Landscape Analysis
Carolin Benjamins · Anja Jankovic · Elena Raponi · Koen van der Blom · Marius Lindauer · Carola Doerr

@
in NeurIPS 2022 Workshop on Meta-Learning »
Event URL: https://openreview.net/forum?id=cmxtTF_IHd »

Bayesian optimization (BO) algorithms form a class of surrogate-based heuristics, aimed at efficiently computing high-quality solutions for numerical black-box optimization problems. The BO pipeline is highly modular, with different design choices for the initial sampling strategy, the surrogate model, the acquisition function (AF), the solver used to optimize the AF, etc. We demonstrate in this work that a dynamic selection of the AF can benefit the BO design. More precisely, we show that already a naive random forest regression model, built on top of exploratory landscape analysis features that are computed from the initial design points, suffices to recommend AFs that outperform any static choice, when considering performance over the classic BBOB benchmark suite for derivative-free numerical optimization methods on the COCO platform. Our work hence paves a way towards AutoML-assisted, on-the-fly BO designs that adjust their behavior on a run-by-run basis.

Author Information

Carolin Benjamins (Leibniz University Hanover)
Anja Jankovic (Sorbonne University)
Elena Raponi (Technical University of Munich, Sorbonne Université)

Elena Raponi is a Postdoctoral Researcher at the Technical University of Munich, Chair of Computational Mechanics, and Sorbonne University, LIP6 Department. Her position is funded by a DAAD Prime Postdoctoral Fellowship, which was awarded in December 2021. Previously, she was a postdoctoral fellow in the Natural Computing Research Group at the Leiden Institute of Advanced Computer Science. She received her Ph.D. in Applied Mathematics from the University of Camerino, Italy, in May 2021. Her research mainly focuses on surrogate-based optimization in continuous domains, high-dimensional Bayesian optimization, and analytical and numerical modeling techniques for the optimization of geometries and materials in structural mechanics. She has published 1 book chapter, 5 conference papers, and 9 journal articles in reputed journals such as Computer Methods in Applied Mechanics and Engineering, Composite Structures, and Reports on Progress in Physics. She is a co-developer of the open-source package [GSAreport](https://github.com/Basvanstein/GSAreport) and contributed to the development of the library [Bayesian-Optimization](https://github.com/wangronin/Bayesian-Optimization).

Koen van der Blom (Sorbonne University)
Koen van der Blom

Koen van der Blom is a post-doctoral researcher at the LIP6 of Sorbonne Universite in Paris, France. He received his Ph.D. in 2019 on multi-objective evolutionary optimisation for early-stage building design from Leiden University (the Netherlands), and his thesis received an honourable mention for the 2020 ACM SIGEVO Best Dissertation Award. His research interests include the accessibility of meta-algorithms such as automated algorithm selection and configuration, evolutionary computation for real-world applications, and multi-objective optimisation.

Marius Lindauer (Leibniz Universität Hannover)
Carola Doerr (CNRS & Sorbonne University)

More from the Same Authors