GraphSAINT: Graph Sampling Based Inductive Learning Method
Graph Convolutional Networks (GCNs) are powerful models for learning representations of attributed graphs. To scale GCNs to large graphs, state-of-the-art methods use various layer sampling techniques to alleviate the "neighbor explosion" problem during minibatch training. We propose GraphSAINT, a graph sampling based inductive learning method that improves training efficiency and accuracy in a fundamentally different way. By changing perspective, GraphSAINT constructs minibatches by sampling the training graph, rather than the nodes or edges across GCN layers. Each iteration, a complete GCN is built from the properly sampled subgraph. Thus, we ensure fixed number of well-connected nodes in all layers. We further propose normalization technique to eliminate bias, and sampling algorithms for variance reduction. Importantly, we can decouple the sampling from the forward and backward propagation, and extend GraphSAINT with many architecture variants (e.g., graph attention, jumping connection). GraphSAINT demonstrates superior performance in both accuracy and training time on five large graphs, and achieves new state-of-the-art F1 scores for PPI (0.995) and Reddit (0.970).
Code (8)
Tasks
Graph AttentionGraph EmbeddingGraph Representation LearningGraph SamplingInductive LearningNode ClassificationNode Property PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Accurate, Efficient and Scalable Training of Graph Neural Networks
Graph Neural Networks (GNNs) are powerful deep learning models to generate node embeddings on graphs. When applying deep GNNs on large graphs, it is still challenging to perform training in an efficient and scalable way.…
Graph SamplingPolicyClusterGCN: Identifying Efficient Clusters for Training Graph Convolutional Networks
Graph convolutional networks (GCNs) have achieved huge success in several machine learning (ML) tasks on graph-structured data. Recently, several sampling techniques have been proposed for the efficient training of GCNs …
graph partitioningNode ClassificationReinforcement Learning (RL)The Limits of Graph Samplers for Training Inductive Recommender Systems: Extended results
Inductive Recommender Systems are capable of recommending for new users and with new items thus avoiding the need to retrain after new data reaches the system. However, these methods are still trained on all the data ava…
Graph SamplingRecommendation SystemsBitGNN: Unleashing the Performance Potential of Binary Graph Neural Networks on GPUs
Recent studies have shown that Binary Graph Neural Networks (GNNs) are promising for saving computations of GNNs through binarized tensors. Prior work, however, mainly focused on algorithm designs or training techniques,…
Advancing GraphSAGE with A Data-Driven Node Sampling
As an efficient and scalable graph neural network, GraphSAGE has enabled an inductive capability for inferring unseen nodes or graphs by aggregating subsampled local neighborhoods and by learning in a mini-batch gradient…
General ClassificationGraph Neural NetworkNode ClassificationReinforcement Learning