Timezone: »
Graph neural networks (GNNs) integrate deep architectures and topological structure modeling in an effective way. However, the performance of existing GNNs would decrease significantly when they stack many layers, because of the over-smoothing issue. Node embeddings tend to converge to similar vectors when GNNs keep recursively aggregating the representations of neighbors. To enable deep GNNs, several methods have been explored recently. But they are developed from either techniques in convolutional neural networks or heuristic strategies. There is no generalizable and theoretical principle to guide the design of deep GNNs. To this end, we analyze the bottleneck of deep GNNs by leveraging the Dirichlet energy of node embeddings, and propose a generalizable principle to guide the training of deep GNNs. Based on it, a novel deep GNN framework -- Energetic Graph Neural Networks (EGNN) is designed. It could provide lower and upper constraints in terms of Dirichlet energy at each layer to avoid over-smoothing. Experimental results demonstrate that EGNN achieves state-of-the-art performance by using deep layers.
Author Information
Kaixiong Zhou (Rice University)
Xiao Huang (The Hong Kong Polytechnic University)
Daochen Zha (Texas A&M University)
Rui Chen (Samsung Research America)
Li Li (Samsung)
Soo-Hyun Choi (Samsung Electronics)
Xia Hu (Texas A&M University)
More from the Same Authors
-
2021 : Revisiting Time Series Outlier Detection: Definitions and Benchmarks »
Kwei-Herng Lai · Daochen Zha · Junjie Xu · Yue Zhao · Guanchu Wang · Xia Hu -
2021 Poster: Fairness via Representation Neutralization »
Mengnan Du · Subhabrata Mukherjee · Guanchu Wang · Ruixiang Tang · Ahmed Awadallah · Xia Hu -
2020 Poster: Towards Deeper Graph Neural Networks with Differentiable Group Normalization »
Kaixiong Zhou · Xiao Huang · Yuening Li · Daochen Zha · Rui Chen · Xia Hu -
2020 Poster: Detecting Interactions from Neural Networks via Topological Analysis »
Zirui Liu · Qingquan Song · Kaixiong Zhou · Ting-Hsiang Wang · Ying Shan · Xia Hu