Bootstrapped Representation Learning on Graphs
Current state-of-the-art self-supervised learning methods for graph neural networks are based on contrastive learning. As such, they heavily depend on the construction of augmentations and negative examples. Increasing the number of negative pairs improves performance, thereby requiring quadratic computation and memory cost to achieve peak performance. Inspired by BYOL, a recently introduced method for self-supervised learning that does not require negative pairs, we present Bootstrapped Graph Latents, BGRL, a self-supervised graph representation method that gets rid of this potentially quadratic bottleneck. BGRL outperforms or matches the previous unsupervised state-of-the-art results on several established benchmarks. Moreover, it enables the effective usage of graph attentional (GAT) encoders, allowing us to further improve the state of the art, in particular achieving 70.49% Micro-F1 on the PPI dataset using the linear evaluation protocol.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningLinear evaluationRepresentation LearningSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
WBT-BGRL: A Non-Contrastive Weighted Bipartite Link Prediction Model for Inductive Learning
Link prediction in bipartite graphs is crucial for applications like recommendation systems and failure detection, yet it is less studied than in monopartite graphs. Contrastive methods struggle with inefficient and bias…
Inductive Link PredictionRecommendation SystemsContrastive LearningLarge-Scale Representation Learning on Graphs via Bootstrapping
Self-supervised learning provides a promising path towards eliminating the need for costly label information in representation learning on graphs. However, to achieve state-of-the-art performance, methods often need larg…
Contrastive LearningGraph Representation LearningRepresentation LearningSelf-Supervised LearningAdversarial network embedding with bootstrapped representations for sparse networks
The inherent sparsity of real-world networks presents challenges in learning-rich embeddings and accurately reconstructing networks. To address these challenges, a novel method termed Adversarial Network Embedding with B…
Data VisualizationLink PredictionNetwork EmbeddingNode ClassificationBOURNE: Bootstrapped Self-supervised Learning Framework for Unified Graph Anomaly Detection
Graph anomaly detection (GAD) has gained increasing attention in recent years due to its critical application in a wide range of domains, such as social networks, financial risk management, and traffic analysis. Existing…
Anomaly DetectionCoLAContrastive LearningGraph Anomaly Detection+1Incremental Bootstrapping and Classification of Structured Scenes in a Fuzzy Ontology
We foresee robots that bootstrap knowledge representations and use them for classifying relevant situations and making decisions based on future observations. Particularly for assistive robots, the bootstrapping mechanis…