Timezone: »

 
Poster
Neural Architecture Generator Optimization
Robin Ru · Pedro Esperança · Fabio Maria Carlucci

Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #1035
in Poster Session 4 »

Neural Architecture Search (NAS) was first proposed to achieve state-of-the-art performance through the discovery of new architecture patterns, without human intervention. An over-reliance on expert knowledge in the search space design has however led to increased performance (local optima) without significant architectural breakthroughs, thus preventing truly novel solutions from being reached. In this work we 1) are the first to investigate casting NAS as a problem of finding the optimal network generator and 2) we propose a new, hierarchical and graph-based search space capable of representing an extremely large variety of network types, yet only requiring few continuous hyper-parameters. This greatly reduces the dimensionality of the problem, enabling the effective use of Bayesian Optimisation as a search strategy. At the same time, we expand the range of valid architectures, motivating a multi-objective learning approach. We demonstrate the effectiveness of this strategy on six benchmark datasets and show that our search space generates extremely lightweight yet highly competitive models.

Author Information

Robin Ru (Oxford University)
Pedro Esperança (Huawei Noah's Ark Lab, London)
Fabio Maria Carlucci (Huawei Noah's Ark Lab)

PhD in Computer Vision and Machine Learning

More from the Same Authors