NeurIPS 2019 Expo Demo

Nov. 28, 2022

Expo 2019 Schedule »

Advanced Hyperparameter Optimization Methods

Sponsor: SigOpt

Modelers typically introduce hyperparameter optimization (HPO) toward the end of their model development process. Confining HPO to this “last mile” misses a variety of opportunities to applying HPO techniques throughout the modeling process that can boost modeling outcomes. In this demo, we will provide a prescriptive guide for HPO that includes techniques for using HPO throughout the modeling process and real-world examples of the impact of this approach.

In particular, this demo will include ways to apply advanced HPO methods to address:

-Metric definition, selection and optimization -Data and feature transformation parameter tuning -Architecture tuning as part of hyperparameter optimization -Deep learning model convergence with automated early stopping -Long training cycles with multitask tuning and parallelism

For each of these use cases, the team will share more detail around the research underpinning for these HPO techniques as well as an applied use case that contextualizes how to incorporate it into the modeling process. The goal is for this combination to offer useful information for modelers, modeling team leaders and modeling platform engineers.