Timezone: »
In recent years, generative adversarial networks (GANs) have been an actively studied topic and shown to successfully produce high-quality realistic images in various domains. The controllable synthesis ability of GAN generators suggests that they maintain informative, disentangled, and explainable image representations, but leveraging and transferring their representations to downstream tasks is largely unexplored. In this paper, we propose to distill knowledge from GAN generators by squeezing and spanning their representations. We \emph{squeeze} the generator features into representations that are invariant to semantic-preserving transformations through a network before they are distilled into the student network. We \emph{span} the distilled representation of the synthetic domain to the real domain by also using real training data to remedy the mode collapse of GANs and boost the student network performance in a real domain. Experiments justify the efficacy of our method and reveal its great significance in self-supervised representation learning. Code is available at https://github.com/yangyu12/squeeze-and-span.
Author Information
Yu Yang (Tsinghua University, Tsinghua University)
Xiaotian Cheng (Tsinghua University, Tsinghua University)
Chang Liu (Tsinghua University, Tsinghua University)
Hakan Bilen (University of Edinburgh)
Xiangyang Ji (Tsinghua University)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 Poster: Distilling Representations from GAN Generator via Squeeze and Span »
Tue. Nov 29th 05:00 -- 07:00 PM Room Hall J #630
More from the Same Authors
-
2022 Poster: Near-Optimal Regret Bounds for Multi-batch Reinforcement Learning »
Zihan Zhang · Yuhang Jiang · Yuan Zhou · Xiangyang Ji -
2022 Poster: Self-Organized Group for Cooperative Multi-agent Reinforcement Learning »
Jianzhun Shao · Zhiqiang Lou · Hongchang Zhang · Yuhang Jiang · Shuncheng He · Xiangyang Ji -
2022 Poster: SPD: Synergy Pattern Diversifying Oriented Unsupervised Multi-agent Reinforcement Learning »
Yuhang Jiang · Jianzhun Shao · Shuncheng He · Hongchang Zhang · Xiangyang Ji -
2022 : An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation »
Ziquan Liu · Yi Xu · Yuanhong Xu · Qi Qian · Hao Li · Rong Jin · Xiangyang Ji · Antoni Chan -
2023 Poster: Discover and Align Taxonomic Context Priors for Open-world Semi-Supervised Learning »
Yu Wang · Zhun Zhong · Pengchong Qiao · Xuxin Cheng · Xiawu Zheng · Chang Liu · Nicu Sebe · Rongrong Ji · Jie Chen -
2023 Poster: Supported Value Regularization for Offline Reinforcement Learning »
Yixiu Mao · Hongchang Zhang · Chen Chen · Yi Xu · Xiangyang Ji -
2023 Poster: DDF-HO: Hand-Held Object Reconstruction via Conditional Directed Distance Field »
Chenyangguang Zhang · Yan Di · Ruida Zhang · Guangyao Zhai · Fabian Manhardt · Federico Tombari · Xiangyang Ji -
2023 Poster: Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learning »
Jianzhun Shao · yun qu · Chen Chen · Hongchang Zhang · Xiangyang Ji -
2023 Poster: Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning Benchmarks »
yun qu · Boyuan Wang · Jianzhun Shao · Yuhang Jiang · Chen Chen · Zhenbin Ye · Liu Linc · Yang Feng · Lin Lai · Hongyang Qin · Minwen Deng · Juchao Zhuo · Deheng Ye · Qiang Fu · YANG GUANG · Wei Yang · Lanxiao Huang · Xiangyang Ji -
2022 Spotlight: Lightning Talks 5A-3 »
Minting Pan · Xiang Chen · Wenhan Huang · Can Chang · Zhecheng Yuan · Jianzhun Shao · Yushi Cao · Peihao Chen · Ke Xue · Zhengrong Xue · Zhiqiang Lou · Xiangming Zhu · Lei Li · Zhiming Li · Kai Li · Jiacheng Xu · Dongyu Ji · Ni Mu · Kun Shao · Tianpei Yang · Kunyang Lin · Ningyu Zhang · Yunbo Wang · Lei Yuan · Bo Yuan · Hongchang Zhang · Jiajun Wu · Tianze Zhou · Xueqian Wang · Ling Pan · Yuhang Jiang · Xiaokang Yang · Xiaozhuan Liang · Hao Zhang · Weiwen Hu · Miqing Li · YAN ZHENG · Matthew Taylor · Huazhe Xu · Shumin Deng · Chao Qian · YI WU · Shuncheng He · Wenbing Huang · Chuanqi Tan · Zongzhang Zhang · Yang Gao · Jun Luo · Yi Li · Xiangyang Ji · Thomas Li · Mingkui Tan · Fei Huang · Yang Yu · Huazhe Xu · Dongge Wang · Jianye Hao · Chuang Gan · Yang Liu · Luo Si · Hangyu Mao · Huajun Chen · Jianye Hao · Jun Wang · Xiaotie Deng -
2022 Spotlight: Self-Organized Group for Cooperative Multi-agent Reinforcement Learning »
Jianzhun Shao · Zhiqiang Lou · Hongchang Zhang · Yuhang Jiang · Shuncheng He · Xiangyang Ji -
2022 Spotlight: Lightning Talks 2A-3 »
David Buterez · Chengan He · Xuan Kan · Yutong Lin · Konstantin Schürholt · Yu Yang · Louis Annabi · Wei Dai · Xiaotian Cheng · Alexandre Pitti · Ze Liu · Jon Paul Janet · Jun Saito · Boris Knyazev · Mathias Quoy · Zheng Zhang · James Zachary · Steven J Kiddle · Xavier Giro-i-Nieto · Chang Liu · Hejie Cui · Zilong Zhang · Hakan Bilen · Damian Borth · Dino Oglic · Holly Rushmeier · Han Hu · Xiangyang Ji · Yi Zhou · Nanning Zheng · Ying Guo · Pietro Liò · Stephen Lin · Carl Yang · Yue Cao -
2022 Spotlight: Lightning Talks 1B-4 »
Andrei Atanov · Shiqi Yang · Wanshan Li · Yongchang Hao · Ziquan Liu · Jiaxin Shi · Anton Plaksin · Jiaxiang Chen · Ziqi Pan · yaxing wang · Yuxin Liu · Stepan Martyanov · Alessandro Rinaldo · Yuhao Zhou · Li Niu · Qingyuan Yang · Andrei Filatov · Yi Xu · Liqing Zhang · Lili Mou · Ruomin Huang · Teresa Yeo · kai wang · Daren Wang · Jessica Hwang · Yuanhong Xu · Qi Qian · Hu Ding · Michalis Titsias · Shangling Jui · Ajay Sohmshetty · Lester Mackey · Joost van de Weijer · Hao Li · Amir Zamir · Xiangyang Ji · Antoni Chan · Rong Jin -
2022 Spotlight: Improved Fine-Tuning by Better Leveraging Pre-Training Data »
Ziquan Liu · Yi Xu · Yuanhong Xu · Qi Qian · Hao Li · Xiangyang Ji · Antoni Chan · Rong Jin -
2022 : Synthesizing Informative Training Samples with GAN »
Bo Zhao · Hakan Bilen -
2022 Poster: Improved Fine-Tuning by Better Leveraging Pre-Training Data »
Ziquan Liu · Yi Xu · Yuanhong Xu · Qi Qian · Hao Li · Xiangyang Ji · Antoni Chan · Rong Jin -
2021 Poster: Improved Variance-Aware Confidence Sets for Linear Bandits and Linear Mixture MDP »
Zihan Zhang · Jiaqi Yang · Xiangyang Ji · Simon Du -
2021 Poster: TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification »
Zhuchen Shao · Hao Bian · Yang Chen · Yifeng Wang · Jian Zhang · Xiangyang Ji · yongbing zhang -
2020 Poster: Almost Optimal Model-Free Reinforcement Learningvia Reference-Advantage Decomposition »
Zihan Zhang · Yuan Zhou · Xiangyang Ji -
2019 Poster: Regret Minimization for Reinforcement Learning by Evaluating the Optimal Bias Function »
Zihan Zhang · Xiangyang Ji -
2018 Poster: Modelling and unsupervised learning of symmetric deformable object categories »
James Thewlis · Hakan Bilen · Andrea Vedaldi -
2018 Poster: Unsupervised Learning of Object Landmarks through Conditional Image Generation »
Tomas Jakab · Ankush Gupta · Hakan Bilen · Andrea Vedaldi -
2017 Poster: Learning multiple visual domains with residual adapters »
Sylvestre-Alvise Rebuffi · Hakan Bilen · Andrea Vedaldi -
2017 Spotlight: Learning multiple visual domains with residual adapters »
Sylvestre-Alvise Rebuffi · Hakan Bilen · Andrea Vedaldi -
2017 Poster: Unsupervised learning of object frames by dense equivariant image labelling »
James Thewlis · Hakan Bilen · Andrea Vedaldi -
2017 Oral: Unsupervised learning of object frames by dense equivariant image labelling »
James Thewlis · Hakan Bilen · Andrea Vedaldi -
2016 Poster: Integrated perception with recurrent multi-task neural networks »
Hakan Bilen · Andrea Vedaldi