Timezone: »

 
Poster
Rethinking Neural Operations for Diverse Tasks
Nicholas Roberts · Mikhail Khodak · Tri Dao · Liam Li · Christopher Ré · Ameet Talwalkar

Tue Dec 07 04:30 PM -- 06:00 PM (PST) @
in Poster Session 2 »

An important goal of AutoML is to automate-away the design of neural networks on new tasks in under-explored domains. Motivated by this goal, we study the problem of enabling users to discover the right neural operations given data from their specific domain. We introduce a search space of operations called XD-Operations that mimic the inductive bias of standard multi-channel convolutions while being much more expressive: we prove that it includes many named operations across multiple application areas. Starting with any standard backbone such as ResNet, we show how to transform it into a search space over XD-operations and how to traverse the space using a simple weight sharing scheme. On a diverse set of tasks—solving PDEs, distance prediction for protein folding, and music modeling—our approach consistently yields models with lower error than baseline networks and often even lower error than expert-designed domain-specific approaches.

Keywords: Deep Learning · Machine Learning

Author Information

Nicholas Roberts (University of Wisconsin-Madison)
Mikhail Khodak (CMU)
Tri Dao (Stanford University)
Liam Li (Carnegie Mellon University)
Christopher Ré (Stanford)
Ameet Talwalkar (CMU)

More from the Same Authors