Timezone: »
Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards their transferability. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of EGI (Ego-Graph Information maximization) to analytically achieve this goal. Secondly,when node features are structure-relevant, we conduct an analysis of EGI transferability regarding the difference between the local graph Laplacians of the source and target graphs. We conduct controlled synthetic experiments to directly justify our theoretical conclusions. Comprehensive experiments on two real-world network datasets show consistent results in the analyzed setting of direct-transfering, while those on large-scale knowledge graphs show promising results in the more practical setting of transfering with fine-tuning.
Author Information
Qi Zhu (University of Illinois, Urbana Champaign)
Carl Yang (Emory University)
Yidan Xu (University of Michigan)
Haonan Wang (University of Illinois at Urbana-Champaign)
Chao Zhang (Georgia Institute of Technology)
Jiawei Han (University of Illinois at Urbana-Champaign)
More from the Same Authors
-
2021 Spotlight: Subgraph Federated Learning with Missing Neighbor Generation »
Ke ZHANG · Carl Yang · Xiaoxiao Li · Lichao Sun · Siu Ming Yiu -
2022 : Shift-Robust Node Classification via Graph Clustering Co-training »
Qi Zhu · Chao Zhang · Chanyoung Park · Carl Yang · Jiawei Han -
2022 Spotlight: Lightning Talks 2A-3 »
David Buterez · Chengan He · Xuan Kan · Yutong Lin · Konstantin Schürholt · Yu Yang · Louis Annabi · Wei Dai · Xiaotian Cheng · Alexandre Pitti · Ze Liu · Jon Paul Janet · Jun Saito · Boris Knyazev · Mathias Quoy · Zheng Zhang · James Zachary · Steven J Kiddle · Xavier Giro-i-Nieto · Chang Liu · Hejie Cui · Zilong Zhang · Hakan Bilen · Damian Borth · Dino Oglic · Holly Rushmeier · Han Hu · Xiangyang Ji · Yi Zhou · Nanning Zheng · Ying Guo · Pietro Liò · Stephen Lin · Carl Yang · Yue Cao -
2022 Spotlight: Brain Network Transformer »
Xuan Kan · Wei Dai · Hejie Cui · Zilong Zhang · Ying Guo · Carl Yang -
2022 Poster: UnfoldML: Cost-Aware and Uncertainty-Based Dynamic 2D Prediction for Multi-Stage Classification »
Yanbo Xu · Alind Khare · Glenn Matlin · Monish Ramadoss · Rishikesan Kamaleswaran · Chao Zhang · Alexey Tumanov -
2022 Poster: End-to-end Stochastic Optimization with Energy-based Model »
Lingkai Kong · Jiaming Cui · Yuchen Zhuang · Rui Feng · B. Aditya Prakash · Chao Zhang -
2022 Poster: Generating Training Data with Language Models: Towards Zero-Shot Language Understanding »
Yu Meng · Jiaxin Huang · Yu Zhang · Jiawei Han -
2022 Poster: Brain Network Transformer »
Xuan Kan · Wei Dai · Hejie Cui · Zilong Zhang · Ying Guo · Carl Yang -
2021 Poster: Subgraph Federated Learning with Missing Neighbor Generation »
Ke ZHANG · Carl Yang · Xiaoxiao Li · Lichao Sun · Siu Ming Yiu -
2021 Poster: Universal Graph Convolutional Networks »
Di Jin · Zhizhi Yu · Cuiying Huo · Rui Wang · Xiao Wang · Dongxiao He · Jiawei Han -
2021 Poster: When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting »
Harshavardhan Kamarthi · Lingkai Kong · Alexander Rodriguez · Chao Zhang · B. Aditya Prakash -
2021 Poster: Federated Graph Classification over Non-IID Graphs »
Han Xie · Jing Ma · Li Xiong · Carl Yang -
2021 Poster: Exploiting Data Sparsity in Secure Cross-Platform Social Recommendation »
Jinming Cui · Chaochao Chen · Lingjuan Lyu · Carl Yang · Wang Li -
2021 Poster: Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data »
Qi Zhu · Natalia Ponomareva · Jiawei Han · Bryan Perozzi -
2021 Poster: COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining »
Yu Meng · Chenyan Xiong · Payal Bajaj · saurabh tiwary · Paul Bennett · Jiawei Han · XIA SONG -
2019 Poster: Spherical Text Embedding »
Yu Meng · Jiaxin Huang · Guangyuan Wang · Chao Zhang · Honglei Zhuang · Lance Kaplan · Jiawei Han -
2014 Poster: Robust Tensor Decomposition with Gross Corruption »
Quanquan Gu · Huan Gui · Jiawei Han -
2012 Poster: Selective Labeling via Error Bound Minimization »
Quanquan Gu · Tong Zhang · Chris Ding · Jiawei Han -
2009 Poster: Graph-based Consensus Maximization among Multiple Supervised and Unsupervised Models »
Jing Gao · Feng Liang · Wei Fan · Yizhou Sun · Jiawei Han