Self-Supervised Node Representation Learning via Node-to-Neighbourhood Alignment
Self-supervised node representation learning aims to learn node representations from unlabelled graphs that rival the supervised counterparts. The key towards learning informative node representations lies in how to effectively gain contextual information from the graph structure. In this work, we present simple-yet-effective self-supervised node representation learning via aligning the hidden representations of nodes and their neighbourhood. Our first idea achieves such node-to-neighbourhood alignment by directly maximizing the mutual information between their representations, which, we prove theoretically, plays the role of graph smoothing. Our framework is optimized via a surrogate contrastive loss and a Topology-Aware Positive Sampling (TAPS) strategy is proposed to sample positives by considering the structural dependencies between nodes, which enables offline positive selection. Considering the excessive memory overheads of contrastive learning, we further propose a negative-free solution, where the main contribution is a Graph Signal Decorrelation (GSD) constraint to avoid representation collapse and over-smoothing. The GSD constraint unifies some of the existing constraints and can be used to derive new implementations to combat representation collapse. By applying our methods on top of simple MLP-based node representation encoders, we learn node representations that achieve promising node classification performance on a set of graph-structured datasets from small- to large-scale.
Code (1)
Tasks
Contrastive LearningNode ClassificationRepresentation LearningSimilar Papers 제목 키워드 기반
Node Representation Learning in Graph via Node-to-Neighbourhood Mutual Information Maximization
The key towards learning informative node representations in graphs lies in how to gain contextual information from the neighbourhood. In this work, we present a simple-yet-effective self-supervised node representation l…
Node ClassificationRepresentation LearningDifferential Encoding for Improved Representation Learning over Graphs
Combining the message-passing paradigm with the global attention mechanism has emerged as an effective framework for learning over graphs. The message-passing paradigm and the global attention mechanism fundamentally gen…
Graph Representation LearningRepresentation LearningMemory-Based Neighbourhood Embedding for Visual Recognition
Learning discriminative image feature embeddings is of great importance to visual recognition. To achieve better feature embeddings, most current methods focus on designing different network structures or loss functions,…
Few-Shot LearningImage RetrievalGraphViz2Vec: A Structure-aware Feature Generation Model to Improve Classification in GNNs
GNNs are widely used to solve various tasks including node classification and link prediction. Most of the GNN architectures assume the initial embedding to be random or generated from popular distributions. These initia…
Classificationimage-classificationImage ClassificationLink Prediction+1Locality Preserving Dense Graph Convolutional Networks with Graph Context-Aware Node Representations
Graph convolutional networks (GCNs) have been widely used for representation learning on graph data, which can capture structural patterns on a graph via specifically designed convolution and readout operations. In many …
ClassificationDecoderGeneral ClassificationGraph Classification+1