Timezone: »
Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards their transferability. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of EGI (Ego-Graph Information maximization) to analytically achieve this goal. Secondly,when node features are structure-relevant, we conduct an analysis of EGI transferability regarding the difference between the local graph Laplacians of the source and target graphs. We conduct controlled synthetic experiments to directly justify our theoretical conclusions. Comprehensive experiments on two real-world network datasets show consistent results in the analyzed setting of direct-transfering, while those on large-scale knowledge graphs show promising results in the more practical setting of transfering with fine-tuning.
Author Information
Qi Zhu (University of Illinois, Urbana Champaign)
Carl Yang (Emory University)
Yidan Xu (University of Michigan)
Haonan Wang (University of Illinois at Urbana-Champaign)
Chao Zhang (Georgia Institute of Technology)
Jiawei Han (University of Illinois at Urbana-Champaign)
More from the Same Authors
-
2021 Spotlight: Subgraph Federated Learning with Missing Neighbor Generation »
Ke ZHANG · Carl Yang · Xiaoxiao Li · Lichao Sun · Siu Ming Yiu -
2021 Poster: Subgraph Federated Learning with Missing Neighbor Generation »
Ke ZHANG · Carl Yang · Xiaoxiao Li · Lichao Sun · Siu Ming Yiu -
2021 Poster: Universal Graph Convolutional Networks »
Di Jin · Zhizhi Yu · Cuiying Huo · Rui Wang · Xiao Wang · Dongxiao He · Jiawei Han -
2021 Poster: When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting »
Harshavardhan Kamarthi · Lingkai Kong · Alexander Rodriguez · Chao Zhang · B. Aditya Prakash -
2021 Poster: Federated Graph Classification over Non-IID Graphs »
Han Xie · Jing Ma · Li Xiong · Carl Yang -
2021 Poster: Exploiting Data Sparsity in Secure Cross-Platform Social Recommendation »
Jinming Cui · Chaochao Chen · Lingjuan Lyu · Carl Yang · Wang Li -
2021 Poster: Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data »
Qi Zhu · Natalia Ponomareva · Jiawei Han · Bryan Perozzi -
2021 Poster: COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining »
Yu Meng · Chenyan Xiong · Payal Bajaj · saurabh tiwary · Paul Bennett · Jiawei Han · XIA SONG -
2019 Poster: Spherical Text Embedding »
Yu Meng · Jiaxin Huang · Guangyuan Wang · Chao Zhang · Honglei Zhuang · Lance Kaplan · Jiawei Han -
2014 Poster: Robust Tensor Decomposition with Gross Corruption »
Quanquan Gu · Huan Gui · Jiawei Han -
2012 Poster: Selective Labeling via Error Bound Minimization »
Quanquan Gu · Tong Zhang · Chris Ding · Jiawei Han -
2009 Poster: Graph-based Consensus Maximization among Multiple Supervised and Unsupervised Models »
Jing Gao · Feng Liang · Wei Fan · Yizhou Sun · Jiawei Han