Timezone: »

 
Poster
An Empirical Study on Disentanglement of Negative-free Contrastive Learning
Jinkun Cao · Ruiqian Nai · Qing Yang · Jialei Huang · Yang Gao

Tue Nov 29 02:00 PM -- 04:00 PM (PST) @ Hall J #211
in Poster Session 2 »

Negative-free contrastive learning methods have attracted a lot of attention with simplicity and impressive performances for large-scale pretraining. However, its disentanglement property remains unexplored. In this paper, we examine negative-free contrastive learning methods to study the disentanglement property empirically. We find that existing disentanglement metrics fail to make meaningful measurements for high-dimensional representation models, so we propose a new disentanglement metric based on Mutual Information between latent representations and data factors. With this proposed metric, we benchmark the disentanglement property of negative-free contrastive learning on both popular synthetic datasets and a real-world dataset CelebA. Our study shows that the investigated methods can learn a well-disentangled subset of representation. As far as we know, we are the first to extend the study of disentangled representation learning to high-dimensional representation space and introduce negative-free contrastive learning methods into this area. The source code of this paper is available at https://github.com/noahcao/disentanglementlibmed.

Author Information

Jinkun Cao (Carnegie Mellon University)
Ruiqian Nai (Tsinghua University, Tsinghua University)
Qing Yang (Shanghai Jiao Tong University)
Jialei Huang (Tsinghua University, Tsinghua University)
Yang Gao (Tsinghua University)

More from the Same Authors