Timezone: »

 
Poster
Topic Modeling Revisited: A Document Graph-based Neural Network Perspective
Dazhong Shen · Chuan Qin · Chao Wang · Zheng Dong · Hengshu Zhu · Hui Xiong

Wed Dec 08 04:30 PM -- 06:00 PM (PST) @
in Poster Session 4 »

Most topic modeling approaches are based on the bag-of-words assumption, where each word is required to be conditionally independent in the same document. As a result, both of the generative story and the topic formulation have totally ignored the semantic dependency among words, which is important for improving the semantic comprehension and model interpretability. To this end, in this paper, we revisit the task of topic modeling by transforming each document into a directed graph with word dependency as edges between word nodes, and develop a novel approach, namely Graph Neural Topic Model (GNTM). Specifically, in GNTM, a well-defined probabilistic generative story is designed to model both the graph structure and word sets with multinomial distributions on the vocabulary and word dependency edge set as the topics. Meanwhile, a Neural Variational Inference (NVI) approach is proposed to learn our model with graph neural networks to encode the document graphs. Besides, we theoretically demonstrate that Latent Dirichlet Allocation (LDA) can be derived from GNTM as a special case with similar objective functions. Finally, extensive experiments on four benchmark datasets have clearly demonstrated the effectiveness and interpretability of GNTM compared with state-of-the-art baselines.

Keywords: Deep Learning · Generative Model · Graph Learning · Interpretability

Author Information

Dazhong Shen (University of Science and Technology of China)
Chuan Qin (University of Science and Technology of China,)
Chao Wang (University of Science and Technology of China)
Zheng Dong (University of British Columbia)
Hengshu Zhu (Baidu)
Hui Xiong

More from the Same Authors