Timezone: »
A good visual representation is an inference map from observations (images) to features (vectors) that faithfully reflects the hidden modularized generative factors (semantics). In this paper, we formulate the notion of "good" representation from a group-theoretic view using Higgins' definition of disentangled representation, and show that existing Self-Supervised Learning (SSL) only disentangles simple augmentation features such as rotation and colorization, thus unable to modularize the remaining semantics. To break the limitation, we propose an iterative SSL algorithm: Iterative Partition-based Invariant Risk Minimization (IP-IRM), which successfully grounds the abstract semantics and the group acting on them into concrete contrastive learning. At each iteration, IP-IRM first partitions the training samples into two subsets that correspond to an entangled group element. Then, it minimizes a subset-invariant contrastive loss, where the invariance guarantees to disentangle the group element. We prove that IP-IRM converges to a fully disentangled representation and show its effectiveness on various benchmarks. Codes are available at https://github.com/Wangt-CN/IP-IRM.
Author Information
Tan Wang (Nanyang Technological University)
Zhongqi Yue (Nanyang Technological University)
Jianqiang Huang (Damo Academy, Alibaba Group)
Qianru Sun (Singapore Management University)
Hanwang Zhang (NTU)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Self-Supervised Learning Disentangled Group Representation as Feature »
Dates n/a. Room
More from the Same Authors
-
2022 Poster: Respecting Transfer Gap in Knowledge Distillation »
Yulei Niu · Long Chen · Chang Zhou · Hanwang Zhang -
2021 Poster: RMM: Reinforced Memory Management for Class-Incremental Learning »
Yaoyao Liu · Bernt Schiele · Qianru Sun -
2021 Poster: How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness? »
Xinshuai Dong · Anh Tuan Luu · Min Lin · Shuicheng Yan · Hanwang Zhang -
2021 Poster: Introspective Distillation for Robust Question Answering »
Yulei Niu · Hanwang Zhang -
2020 Poster: Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect »
Kaihua Tang · Jianqiang Huang · Hanwang Zhang -
2020 Poster: Causal Intervention for Weakly-Supervised Semantic Segmentation »
Dong Zhang · Hanwang Zhang · Jinhui Tang · Xian-Sheng Hua · Qianru Sun -
2020 Oral: Causal Intervention for Weakly-Supervised Semantic Segmentation »
Dong Zhang · Hanwang Zhang · Jinhui Tang · Xian-Sheng Hua · Qianru Sun -
2020 Poster: Interventional Few-Shot Learning »
Zhongqi Yue · Hanwang Zhang · Qianru Sun · Xian-Sheng Hua -
2019 Poster: Learning to Self-Train for Semi-Supervised Few-Shot Classification »
Xinzhe Li · Qianru Sun · Yaoyao Liu · Qin Zhou · Shibao Zheng · Tat-Seng Chua · Bernt Schiele -
2018 Poster: Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks »
Hang Gao · Zheng Shou · Alireza Zareian · Hanwang Zhang · Shih-Fu Chang -
2017 Poster: Pose Guided Person Image Generation »
Liqian Ma · Xu Jia · Qianru Sun · Bernt Schiele · Tinne Tuytelaars · Luc Van Gool