Spectral Augmentation for Self-Supervised Learning on Graphs
Graph contrastive learning (GCL), as an emerging self-supervised learning technique on graphs, aims to learn representations via instance discrimination. Its performance heavily relies on graph augmentation to reflect invariant patterns that are robust to small perturbations; yet it still remains unclear about what graph invariance GCL should capture. Recent studies mainly perform topology augmentations in a uniformly random manner in the spatial domain, ignoring its influence on the intrinsic structural properties embedded in the spectral domain. In this work, we aim to find a principled way for topology augmentations by exploring the invariance of graphs from the spectral perspective. We develop spectral augmentation which guides topology augmentations by maximizing the spectral change. Extensive experiments on both graph and node classification tasks demonstrate the effectiveness of our method in self-supervised representation learning. The proposed method also brings promising generalization capability in transfer learning, and is equipped with intriguing robustness property under adversarial attacks. Our study sheds light on a general principle for graph topology augmentation.
Code (1)
Tasks
Contrastive LearningNode ClassificationRepresentation LearningSelf-Supervised LearningTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Augmentation-Free Self-Supervised Learning on Graphs
Inspired by the recent success of self-supervised methods applied on images, self-supervised learning on graph structured data has seen rapid growth especially centered on augmentation-based contrastive methods. However,…
Node ClassificationSelf-Supervised LearningRethinking Spectral Augmentation for Contrast-based Graph Self-Supervised Learning
The recent surge in contrast-based graph self-supervised learning has prominently featured an intensified exploration of spectral cues. Spectral augmentation, which involves modifying a graph's spectral properties such a…
Self-Supervised LearningAugmentation-Free Graph Contrastive Learning with Performance Guarantee
Graph contrastive learning (GCL) is the most representative and prevalent self-supervised learning approach for graph-structured data. Despite its remarkable success, existing GCL methods highly rely on an augmentation s…
Contrastive LearningGraph Neural NetworkSelf-Supervised LearningSelf-Supervised Graph Learning via Spectral Bootstrapping and Laplacian-Based Augmentations
We present LaplaceGNN, a novel self-supervised graph learning framework that bypasses the need for negative sampling by leveraging spectral bootstrapping techniques. Our method integrates Laplacian-based signals into the…
Graph LearningProvable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Recent works in self-supervised learning have advanced the state-of-the-art by relying on the contrastive learning paradigm, which learns representations by pushing positive pairs, or similar examples from the same class…
Contrastive LearningGeneralization BoundsSelf-Supervised Learning