NeurIPS 2020

Timezone: »

Poster

Passport-aware Normalization for Deep Model Protection

Jie Zhang · Dongdong Chen · Jing Liao · Weiming Zhang · Gang Hua · Nenghai Yu

Thu Dec 10 09:00 PM -- 11:00 PM (PST) @ Poster Session 6 #1868

in Poster Session 7 »

Despite tremendous success in many application scenarios, deep learning faces serious intellectual property (IP) infringement threats. Considering the cost of designing and training a good model, infringements will significantly infringe the interests of the original model owner. Recently, many impressive works have emerged for deep model IP protection. However, they either are vulnerable to ambiguity attacks, or require changes in the target network structure by replacing its original normalization layers and hence cause significant performance drops. To this end, we propose a new passport-aware normalization formulation, which is generally applicable to most existing normalization layers and only needs to add another passport-aware branch for IP protection. This new branch is jointly trained with the target model but discarded in the inference stage. Therefore it causes no structure change in the target model. Only when the model IP is suspected to be stolen by someone, the private passport-aware branch is added back for ownership verification. Through extensive experiments, we verify its effectiveness in both image and 3D point recognition models. It is demonstrated to be robust not only to common attack techniques like fine-tuning and model compression, but also to ambiguity attacks. By further combining it with trigger-set based methods, both black-box and white-box verification can be achieved for enhanced security of deep learning models deployed in real systems.

Author Information

Jie Zhang (University of Science and Technology of China)

Dongdong Chen (Microsoft Cloud AI)

Jing Liao (City University of Hong Kong)

Weiming Zhang (University of Science and Technology of China)

Gang Hua (Wormpex AI Research)

Gang Hua is the Vice President and Chief Scientist of Wormpex AI Research. His research focuses on computer vision, pattern recognition, machine learning, robotics, towards general Artificial Intelligence, with primary applications in cloud and edge intelligence, and currently with a focus on new retail intelligence. Before that, he served in various roles at Microsoft (2015-18) as the Science/Technical Adviser to the CVP of the Computer Vision Group, Director of Computer Vision Science Team in Redmond and Taipei ATL, and Senior Principal Researcher/Research Manager at Microsoft Research . He was an Associate Professor at Stevens Institute of Technology (2011-15). During 2014-15, he took an on leave and worked on the Amazon-Go project. He was a Visiting Researcher (2011-14) and a Research Staff Member (2010-11) at IBM Research T. J. Watson Center, a Senior Researcher (2009-10) at Nokia Research Center Hollywood, and a Senior Scientist (2006-09) at Microsoft Live labs Research. He received his Ph.D. degree in ECE from Northwestern University in 2006. He is an IEEE Fellow, an IAPR Fellow, and an ACM Distinguished Scientist. He is the receipient of the 2015 IAPR Young Biometrics Investigator Award. He has published more than 150 peer reviewed papers in top conferences and journals. To date, he holds 19 US patents and has 15 more patents pending. (See https://www.linkedin.com/in/gang-hua-87aa22a/ for my professional profile.)