A Simplified Framework for Contrastive Learning for Node Representations
Contrastive learning has recently established itself as a powerful self-supervised learning framework for extracting rich and versatile data representations. Broadly speaking, contrastive learning relies on a data augmentation scheme to generate two versions of the input data and learns low-dimensional representations by maximizing a normalized temperature-scaled cross entropy loss (NT-Xent) to identify augmented samples corresponding to the same original entity. In this paper, we investigate the potential of deploying contrastive learning in combination with Graph Neural Networks for embedding nodes in a graph. Specifically, we show that the quality of the resulting embeddings and training time can be significantly improved by a simple column-wise postprocessing of the embedding matrix, instead of the row-wise postprocessing via multilayer perceptrons (MLPs) that is adopted by the majority of peer methods. This modification yields improvements in downstream classification tasks of up to 1.5% and even beats existing state-of-the-art approaches on 6 out of 8 different benchmarks. We justify our choices of postprocessing by revisiting the "alignment vs. uniformity paradigm", and show that column-wise post-processing improves both "alignment" and "uniformity" of the embeddings.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningData AugmentationSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GraphCL: Contrastive Self-Supervised Learning of Graph Representations
We propose Graph Contrastive Learning (GraphCL), a general framework for learning node representations in a self supervised manner. GraphCL learns node embeddings by maximizing the similarity between the representations …
Contrastive LearningInductive LearningNode ClassificationSelf-Supervised LearningEliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learning
Graph Contrastive Learning (GCL) has recently drawn much research interest for learning generalizable node representations in a self-supervised manner. In general, the contrastive learning process in GCL is performed on …
Contrastive LearningGraph Neural NetworkRepresentation LearningImGCL: Revisiting Graph Contrastive Learning on Imbalanced Node Classification
Graph contrastive learning (GCL) has attracted a surge of attention due to its superior performance for learning node/graph representations without labels. However, in practice, the underlying class distribution of unlab…
ClassificationContrastive LearningNode ClassificationOnline ClusteringBayesian Robust Graph Contrastive Learning
Graph Neural Networks (GNNs) have been widely used to learn node representations and with outstanding performance on various tasks such as node classification. However, noise, which inevitably exists in real-world graph …
Contrastive LearningNode ClassificationConCur: Self-supervised graph representation based on contrastive learning with curriculum negative sampling
Contrastive learning has made breakthrough advancements in graph representation learning, which encourages the representation of positive samples to be close and those of negative samples to be far away. However, existin…
Contrastive LearningGraph Representation LearningNode ClassificationRepresentation Learning+1