Ripple Walk Training: A Subgraph-based training framework for Large and Deep Graph Neural Network
Graph neural networks (GNNs) have achieved outstanding performance in learning graph-structured data and various tasks. However, many current GNNs suffer from three common problems when facing large-size graphs or using a deeper structure: neighbors explosion, node dependence, and oversmoothing. Such problems attribute to the data structures of the graph itself or the designing of the multi-layers GNNs framework, and can lead to low training efficiency and high space complexity. To deal with these problems, in this paper, we propose a general subgraph-based training framework, namely Ripple Walk Training (RWT), for deep and large graph neural networks. RWT samples subgraphs from the full graph to constitute a mini-batch, and the full GNN is updated based on the mini-batch gradient. We analyze the high-quality subgraphs to train GNNs in a theoretical way. A novel sampling method Ripple Walk Sampler works for sampling these high-quality subgraphs to constitute the mini-batch, which considers both the randomness and connectivity of the graph-structured data. Extensive experiments on different sizes of graphs demonstrate the effectiveness and efficiency of RWT in training various GNNs (GCN & GAT).
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeGraph Neural NetworkSimilar Papers 제목 키워드 기반
SUREL+: Moving from Walks to Sets for Scalable Subgraph-based Graph Representation Learning
Subgraph-based graph representation learning (SGRL) has recently emerged as a powerful tool in many prediction tasks on graphs due to its advantages in model expressiveness and generalization ability. Most previous SGRL …
Graph Representation LearningPredictionRepresentation LearningAlgorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning
Subgraph-based graph representation learning (SGRL) has been recently proposed to deal with some fundamental challenges encountered by canonical graph neural networks (GNNs), and has demonstrated advantages in many impor…
Graph Representation LearningPredictionRepresentation LearningAdaptive Embedded Subgraph Algorithms using Walk-Sum Analysis
We consider the estimation problem in Gaussian graphical models with arbitrary structure. We analyze the Embedded Trees algorithm, which solves a sequence of problems on tractable subgraphs thereby leading to the solutio…
Self-supervised Subgraph Neural Network With Deep Reinforcement Walk Exploration
Graph data, with its structurally variable nature, represents complex real-world phenomena like chemical compounds, protein structures, and social networks. Traditional Graph Neural Networks (GNNs) primarily utilize the …
Training Differentially Private Graph Neural Networks with Random Walk Sampling
Deep learning models are known to put the privacy of their training data at risk, which poses challenges for their safe and ethical release to the public. Differentially private stochastic gradient descent is the de fact…
Privacy PreservingPrivacy Preserving Deep Learning