Timezone: »
In this paper, we consider domain-invariant deep learning by explicitly modeling domain shifts with only a small amount of domain-specific parameters in a Convolutional Neural Network (CNN). By exploiting the observation that a convolutional filter can be well approximated as a linear combination of a small set of dictionary atoms, we show for the first time, both empirically and theoretically, that domain shifts can be effectively handled by decomposing a convolutional layer into a domain-specific atom layer and a domain-shared coefficient layer, while both remain convolutional. An input channel will now first convolve spatially only with each respective domain-specific dictionary atom to ``absorb" domain variations, and then output channels are linearly combined using common decomposition coefficients trained to promote shared semantics across domains. We use toy examples, rigorous analysis, and real-world examples with diverse datasets and architectures, to show the proposed plug-in framework's effectiveness in cross and joint domain performance and domain adaptation. With the proposed architecture, we need only a small set of dictionary atoms to model each additional domain, which brings a negligible amount of additional parameters, typically a few hundred.
Author Information
Ze Wang (Purdue University)
Xiuyuan Cheng (Duke University)
Guillermo Sapiro (Duke University)
Qiang Qiu (Purdue University)
More from the Same Authors
-
2021 Spotlight: Image Generation using Continuous Filter Atoms »
Ze Wang · Seunghyun Hwang · Zichen Miao · Qiang Qiu -
2021 : Computer Vision Analysis of Caregiver-Child Interactions in Children with Neurodevelopmental Disorders »
Dmitry Isaev · J. Matias Di Martino · Kimberley Carpenter · Guillermo Sapiro · geraldine Dawson -
2022 : Invertible Neural Networks for Graph Prediction »
Chen Xu · Xiuyuan Cheng · Yao Xie -
2021 Poster: Learning to Learn Dense Gaussian Processes for Few-Shot Learning »
Ze Wang · Zichen Miao · Xiantong Zhen · Qiang Qiu -
2021 Poster: Neural Tangent Kernel Maximum Mean Discrepancy »
Xiuyuan Cheng · Yao Xie -
2021 Poster: Spatiotemporal Joint Filter Decomposition in 3D Convolutional Neural Networks »
Zichen Miao · Ze Wang · Xiuyuan Cheng · Qiang Qiu -
2021 Poster: Image Generation using Continuous Filter Atoms »
Ze Wang · Seunghyun Hwang · Zichen Miao · Qiang Qiu -
2021 Poster: Exploiting a Zoo of Checkpoints for Unseen Tasks »
Jiaji Huang · Qiang Qiu · Kenneth Church -
2020 Poster: Learning to Learn Variational Semantic Memory »
Xiantong Zhen · Yingjun Du · Huan Xiong · Qiang Qiu · Cees Snoek · Ling Shao -
2020 Poster: Instance-based Generalization in Reinforcement Learning »
Martin Bertran · Natalia Martinez · Mariano Phielipp · Guillermo Sapiro -
2015 Poster: Discriminative Robust Transformation Learning »
Jiaji Huang · Qiang Qiu · Guillermo Sapiro · Robert Calderbank