Rethinking and Simplifying Bootstrapped Graph Latents
Graph contrastive learning (GCL) has emerged as a representative paradigm in graph self-supervised learning, where negative samples are commonly regarded as the key to preventing model collapse and producing distinguishable representations. Recent studies have shown that GCL without negative samples can achieve state-of-the-art performance as well as scalability improvement, with bootstrapped graph latent (BGRL) as a prominent step forward. However, BGRL relies on a complex architecture to maintain the ability to scatter representations, and the underlying mechanisms enabling the success remain largely unexplored. In this paper, we introduce an instance-level decorrelation perspective to tackle the aforementioned issue and leverage it as a springboard to reveal the potential unnecessary model complexity within BGRL. Based on our findings, we present SGCL, a simple yet effective GCL framework that utilizes the outputs from two consecutive iterations as positive pairs, eliminating the negative samples. SGCL only requires a single graph augmentation and a single graph encoder without additional parameters. Extensive experiments conducted on various graph benchmarks demonstrate that SGCL can achieve competitive performance with fewer parameters, lower time and space costs, and significant convergence speedup.
Code (1)
Tasks
Contrastive LearningSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bootstrapped Representation Learning on Graphs
Current state-of-the-art self-supervised learning methods for graph neural networks are based on contrastive learning. As such, they heavily depend on the construction of augmentations and negative examples. Increasing t…
Contrastive LearningLinear evaluationRepresentation LearningSelf-Supervised LearningWBT-BGRL: A Non-Contrastive Weighted Bipartite Link Prediction Model for Inductive Learning
Link prediction in bipartite graphs is crucial for applications like recommendation systems and failure detection, yet it is less studied than in monopartite graphs. Contrastive methods struggle with inefficient and bias…
Inductive Link PredictionRecommendation SystemsContrastive LearningFeatures Based Adaptive Augmentation for Graph Contrastive Learning
Self-Supervised learning aims to eliminate the need for expensive annotation in graph representation learning, where graph contrastive learning (GCL) is trained with the self-supervision signals containing data-data pair…
Contrastive LearningGraph Representation LearningRepresentation LearningSelf-Supervised LearningBootstrap Latents of Nodes and Neighbors for Graph Self-Supervised Learning
Contrastive learning is a significant paradigm in graph self-supervised learning. However, it requires negative samples to prevent model collapse and learn discriminative representations. These negative samples inevitabl…
Contrastive LearningNode ClassificationNode ClusteringRepresentation Learning+1Large-Scale Representation Learning on Graphs via Bootstrapping
Self-supervised learning provides a promising path towards eliminating the need for costly label information in representation learning on graphs. However, to achieve state-of-the-art performance, methods often need larg…
Contrastive LearningGraph Representation LearningRepresentation LearningSelf-Supervised Learning