Timezone: »

 
Poster
OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal Transport
Zongsheng Cao · Qianqian Xu · Zhiyong Yang · Yuan He · Xiaochun Cao · Qingming Huang

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #703
in Poster Session 1 »

Multi-modal knowledge graph embeddings (KGE) have caught more and more attention in learning representations of entities and relations for link prediction tasks. Different from previous uni-modal KGE approaches, multi-modal KGE can leverage expressive knowledge from a wealth of modalities (image, text, etc.), leading to more comprehensive representations of real-world entities. However, the critical challenge along this course lies in that the multi-modal embedding spaces are usually heterogeneous. In this sense, direct fusion will destroy the inherent spatial structure of different modal embeddings. To overcome this challenge, we revisit multi-modal KGE from a distributional alignment perspective and propose optimal transport knowledge graph embeddings (OTKGE). Specifically, we model the multi-modal fusion procedure as a transport plan moving different modal embeddings to a unified space by minimizing the Wasserstein distance between multi-modal distributions. Theoretically, we show that by minimizing the Wasserstein distance between the individual modalities and the unified embedding space, the final results are guaranteed to maintain consistency and comprehensiveness. Moreover, experimental results on well-established multi-modal knowledge graph completion benchmarks show that our OTKGE achieves state-of-the-art performance.

Keywords: Representation Learning · optimal transport · Multi-modal knowledge graph
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Zongsheng Cao (State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences)
Qianqian Xu (Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences)
Zhiyong Yang (Chinese Academy of Sciences)
Yuan He (Alibaba Group)
Xiaochun Cao (SUN YAT-SEN UNIVERSITY)
Qingming Huang (University of Chinese Academy of Sciences)

More from the Same Authors