Timezone: »

 
Poster
Deep Model Reassembly
Xingyi Yang · Daquan Zhou · Songhua Liu · Jingwen Ye · Xinchao Wang

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #708
in Poster Session 4 »

In this paper, we explore a novel knowledge-transfer task, termed as Deep Model Reassembly (DeRy), for general-purpose model reuse.Given a collection of heterogeneous models pre-trained from distinct sources and with diverse architectures, the goal of DeRy, as its name implies, is to first dissect each model into distinctive building blocks, and then selectively reassemble the derived blocks to produce customized networks under both the hardware resource and performance constraints. Such ambitious nature of DeRy inevitably imposes significant challenges, including, in the first place, the feasibility of its solution. We strive to showcase that, through a dedicated paradigm proposed in this paper, DeRy can be made not only possibly but practically efficiently. Specifically, we conduct the partitions of all pre-trained networks jointly via a cover set optimization, and derive a number of equivalence set, within each of which the network blocks are treated as functionally equivalent and hence interchangeable. The equivalence sets learned in this way, in turn, enable picking and assembling blocks to customize networks subject to certain constraints, which is achieved via solving an integer program backed up with a training-free proxy to estimate the task performance. The reassembled models give rise to gratifying performances with the user-specified constraints satisfied. We demonstrate that on ImageNet, the best reassemble model achieves 78.6% top-1 accuracy without fine-tuning, which could be further elevated to 83.2% with end-to-end fine-tuning. Our code is available at https://github.com/Adamdad/DeRy.

Keywords: representation similarity · Transfer Learning from Model Zoo · Neural Network Reassembly
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Xingyi Yang (National University of Singapore)
Xingyi Yang

Xingyi Yang is a second-year Ph.D student at National University of Singapore(NUS) at Learning and Vision Lab. I am now working under the supervision of Prof.Xinchao Wang.

Daquan Zhou (National University of Singapore)
Songhua Liu (National University of Singapore)
Jingwen Ye (National University of Singapore)
Xinchao Wang

More from the Same Authors