Timezone: »

 
Poster
DM2C: Deep Mixed-Modal Clustering
Yangbangyan Jiang · Qianqian Xu · Zhiyong Yang · Xiaochun Cao · Qingming Huang

Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #63
in Applications -- Computer Vision »

Data exhibited with multiple modalities are ubiquitous in real-world clustering tasks. Most existing methods, however, pose a strong assumption that the pairing information for modalities is available for all instances. In this paper, we consider a more challenging task where each instance is represented in only one modality, which we call mixed-modal data. Without any extra pairing supervision across modalities, it is difficult to find a universal semantic space for all of them. To tackle this problem, we present an adversarial learning framework for clustering with mixed-modal data. Instead of transforming all the samples into a joint modality-independent space, our framework learns the mappings across individual modal spaces by virtue of cycle-consistency. Through these mappings, we could easily unify all the samples into a single modal space and perform the clustering. Evaluations on several real-world mixed-modal datasets could demonstrate the superiority of our proposed framework.

Keywords: Computer Vision · Applications
[ Paper]  [ Poster] 

Author Information

Yangbangyan Jiang (Institute of Information Engineering, Chinese Academy of Sciences)
Qianqian Xu (Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences)
Zhiyong Yang (SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences; SCS, University of Chinese Academy of Sciences)
Xiaochun Cao (Institute of Information Engineering, Chinese Academy of Sciences)
Qingming Huang (University of Chinese Academy of Sciences)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors