Timezone: »

 
Poster
Revisiting Parameter Sharing for Automatic Neural Channel Number Search
Jiaxing Wang · Haoli Bai · Jiaxiang Wu · Xupeng Shi · Junzhou Huang · Irwin King · Michael R Lyu · Jian Cheng

Thu Dec 10 09:00 PM -- 11:00 PM (PST) @ Poster Session 6 #1783
in Poster Session 7 »

Recent advances in neural architecture search inspire many channel number search algorithms~(CNS) for convolutional neural networks. To improve searching efficiency, parameter sharing is widely applied, which reuses parameters among different channel configurations. Nevertheless, it is unclear how parameter sharing affects the searching process. In this paper, we aim at providing a better understanding and exploitation of parameter sharing for CNS. Specifically, we propose affine parameter sharing~(APS) as a general formulation to unify and quantitatively analyze existing channel search algorithms. It is found that with parameter sharing, weight updates of one architecture can simultaneously benefit other candidates. However, it also results in less confidence in choosing good architectures. We thus propose a new strategy of parameter sharing towards a better balance between training efficiency and architecture discrimination. Extensive analysis and experiments demonstrate the superiority of the proposed strategy in channel configuration against many state-of-the-art counterparts on benchmark datasets.

Author Information

Jiaxing Wang (Institute of Automation, Chinese Academy of Sciences)
Haoli Bai (The Chinese University of Hong Kong)
Jiaxiang Wu (Tencent AI Lab)
Xupeng Shi (Northeastern University)
Junzhou Huang (University of Texas at Arlington / Tencent AI Lab)
Irwin King (Chinese University of Hong Kong)
Michael R Lyu (CUHK)
Jian Cheng (Institute of Automation, Chinese Academy of Sciences)

More from the Same Authors