Timezone: »

 
Poster
Bridge the Gap Between Architecture Spaces via A Cross-Domain Predictor
Yuqiao Liu · Yehui Tang · Zeqiong Lv · Yunhe Wang · Yanan Sun

Thu Dec 01 02:00 PM -- 04:00 PM (PST) @ Hall J #629
in Poster Session 6 »

Neural Architecture Search (NAS) can automatically design promising neural architectures without artificial experience. Though it achieves great success, prohibitively high search cost is required to find a high-performance architecture, which blocks its practical implementation. Neural predictor can directly evaluate the performance of neural networks based on their architectures and thereby save much budget. However, existing neural predictors require substantial annotated architectures trained from scratch, which still consume many computational resources. To solve this issue, we propose a Cross-Domain Predictor (CDP), which is trained based on the existing NAS benchmark datasets (e.g., NAS-Bench-101), but can be used to find high-performance architectures in large-scale search spaces. Particularly, we propose a progressive subspace adaptation strategy to address the domain discrepancy between the source architecture space and the target space. Considering the large difference between two architecture spaces, an assistant space is developed to smooth the transfer process. Compared with existing NAS methods, the proposed CDP is much more efficient. For example, CDP only requires the search cost of 0.1 GPU Days to find architectures with 76.9% top-1 accuracy on ImageNet and 97.51% on CIFAR-10.

Keywords: Neural Architecture Search · Domain Adaptation · neural predictor
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Yuqiao Liu (Sichuan University)
Yehui Tang (Peking University)
Zeqiong Lv (Sichuan University)
Yunhe Wang (Huawei Noah's Ark Lab)
Yanan Sun (Sichuan University)

More from the Same Authors