Timezone: »

 
Spotlight
Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum
Nian Liu · Xiao Wang · Deyu Bo · Chuan Shi · Jian Pei

Thu Dec 08 05:00 PM -- 07:00 PM (PST) @
in Lightning Talks 6A-1 »

Graph Contrastive Learning (GCL), learning the node representations by augmenting graphs, has attracted considerable attentions. Despite the proliferation of various graph augmentation strategies, there are still some fundamental questions unclear: what information is essentially learned by GCL? Are there some general augmentation rules behind different augmentations? If so, what are they and what insights can they bring? In this paper, we answer these questions by establishing the connection between GCL and graph spectrum. By an experimental investigation in spectral domain, we firstly find the General grAph augMEntation (GAME) rule for GCL, i.e., the difference of the high-frequency parts between two augmented graphs should be larger than that of low-frequency parts. This rule reveals the fundamental principle to revisit the current graph augmentations and design new effective graph augmentations. Then we theoretically prove that GCL is able to learn the invariance information by contrastive invariance theorem, together with our GAME rule, for the first time, we uncover that the learned representations by GCL essentially encode the low-frequency information, which explains why GCL works. Guided by this rule, we propose a spectral graph contrastive learning module (SpCo), which is a general and GCL-friendly plug-in. We combine it with different existing GCL models, and extensive experiments well demonstrate that it can further improve the performances of a wide variety of different GCL methods.

Author Information

Nian Liu (Beijing University of Post and Telecommunication)
Xiao Wang (Beijing University of Post and Telecommunication)
Deyu Bo (Beijing University of Post and Telecommunication)
Chuan Shi (Beijing University of Post and Telecommunication, Tsinghua University)
Jian Pei (Simon Fraser University)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors