SAIL: Self-Augmented Graph Contrastive Learning
This paper studies learning node representations with graph neural networks (GNNs) for unsupervised scenario. Specifically, we derive a theoretical analysis and provide an empirical demonstration about the non-steady performance of GNNs over different graph datasets, when the supervision signals are not appropriately defined. The performance of GNNs depends on both the node feature smoothness and the locality of graph structure. To smooth the discrepancy of node proximity measured by graph topology and node feature, we proposed SAIL - a novel \underline{S}elf-\underline{A}ugmented graph contrast\underline{i}ve \underline{L}earning framework, with two complementary self-distilling regularization modules, \emph{i.e.}, intra- and inter-graph knowledge distillation. We demonstrate the competitive performance of SAIL on a variety of graph applications. Even with a single GNN layer, SAIL has consistently competitive or even better performance on various benchmark datasets, comparing with state-of-the-art baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningKnowledge DistillationNode ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Self-Augmented In-Context Learning for Unsupervised Word Translation
Recent work has shown that, while large language models (LLMs) demonstrate strong word translation or bilingual lexicon induction (BLI) capabilities in few-shot setups, they still cannot match the performance of 'traditi…
Bilingual Lexicon InductionCross-Lingual Word EmbeddingsFew-Shot LearningIn-Context Learning+8SAIL: Self-Amplified Iterative Learning for Diffusion Model Alignment with Minimal Human Feedback
Aligning diffusion models with human preferences remains challenging, particularly when reward models are unavailable or impractical to obtain, and collecting large-scale preference datasets is prohibitively expensive. \…
SAILViT: Towards Robust and Generalizable Visual Backbones for MLLMs via Gradual Feature Refinement
Vision Transformers (ViTs) are essential as foundation backbones in establishing the visual comprehension capabilities of Multimodal Large Language Models (MLLMs). Although most ViTs achieve impressive performance throug…
Contrastive LearningEmbodied-Symbolic Contrastive Graph Self-Supervised Learning for Molecular Graphs
Dual embodied-symbolic concept representations are the foundation for deep learning and symbolic AI integration. We discuss the use of dual embodied-symbolic concept representations for molecular graph representation lea…
Graph Representation LearningRepresentation LearningSelf-Supervised LearningTensor-Fused Multi-View Graph Contrastive Learning
Graph contrastive learning (GCL) has emerged as a promising approach to enhance graph neural networks' (GNNs) ability to learn rich representations from unlabeled graph-structured data. However, current GCL models face c…
Contrastive LearningGraph Classification