NeurIPS 2022

Timezone: »

PriorBand: HyperBand + Human Expert Knowledge

Neeratyoy Mallik · Carl Hvarfner · Danny Stoll · Maciej Janowski · Edward Bergman · Marius Lindauer · Luigi Nardi · Frank Hutter

in NeurIPS 2022 Workshop on Meta-Learning »

Event URL: https://openreview.net/forum?id=ds21dwfBBH »

Hyperparameters of Deep Learning (DL) pipelines are crucial for their performance. While a large number of methods for hyperparameter optimization (HPO) have been developed, they are misaligned with the desiderata of a modern DL researcher. Since often only a few trials are possible in the development of new DL methods, manual experimentation is still the most prevalent approach to set hyperparameters,relying on the researcher’s intuition and cheap preliminary explorations. To resolve this shortcoming of HPO for DL, we propose PriorBand, an HPO algorithm tailored to DL, able to utilize both expert beliefs and cheap proxy tasks. Empirically, we demonstrate the efficiency of PriorBand across a range of DL models and tasks using as little as the cost of 10 training runs and show its robustness against poor expert beliefs and misleading proxy tasks.

[ Poster] [ OpenReview]

Author Information

Neeratyoy Mallik (University of Freiburg)

Carl Hvarfner (Lund University)

Danny Stoll (University of Freiburg)

First doctoral candidate at the engineering faculty of the University of Freiburg admitted directly after the B.Sc. Working on AutoML, Deep Learning, Neural Architecture Search, Hyperparameter Optimization, Joint Architecture and Hyperparameter Search, Meta-Learning

Maciej Janowski (Albert-Ludwigs-Universität Freiburg)

Edward Bergman (Trinity College Dublin)

Marius Lindauer (Leibniz Universität Hannover)

Luigi Nardi (Lund University and Stanford University)

Frank Hutter (University of Freiburg & Bosch)

Frank Hutter is a Full Professor for Machine Learning at the Computer Science Department of the University of Freiburg (Germany), where he previously was an assistant professor 2013-2017. Before that, he was at the University of British Columbia (UBC) for eight years, for his PhD and postdoc. Frank's main research interests lie in machine learning, artificial intelligence and automated algorithm design. For his 2009 PhD thesis on algorithm configuration, he received the CAIAC doctoral dissertation award for the best thesis in AI in Canada that year, and with his coauthors, he received several best paper awards and prizes in international competitions on machine learning, SAT solving, and AI planning. Since 2016 he holds an ERC Starting Grant for a project on automating deep learning based on Bayesian optimization, Bayesian neural networks, and deep reinforcement learning.