Accelerating Graph Sampling for Graph Machine Learning using GPUs
Representation learning algorithms automatically learn the features of data. Several representation learning algorithms for graph data, such as DeepWalk, node2vec, and GraphSAGE, sample the graph to produce mini-batches that are suitable for training a DNN. However, sampling time can be a significant fraction of training time, and existing systems do not efficiently parallelize sampling. Sampling is an embarrassingly parallel problem and may appear to lend itself to GPU acceleration, but the irregularity of graphs makes it hard to use GPU resources effectively. This paper presents NextDoor, a system designed to effectively perform graph sampling on GPUs. NextDoor employs a new approach to graph sampling that we call transit-parallelism, which allows load balancing and caching of edges. NextDoor provides end-users with a high-level abstraction for writing a variety of graph sampling algorithms. We implement several graph sampling applications, and show that NextDoor runs them orders of magnitude faster than existing systems.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningGPUGraph SamplingRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness
Systems for serving inference requests on graph neural networks (GNN) must combine low latency with high throughout, but they face irregular computation due to skew in the number of sampled graph nodes and aggregated GNN…
CPUGPUGraph SamplingAccelerating SpMM Kernel with Cache-First Edge Sampling for Graph Neural Networks
Graph neural networks (GNNs), an emerging deep learning model class, can extract meaningful representations from highly expressive graph-structured data and are therefore gaining popularity for wider ranges of applicatio…
GPUGSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelism
Graph neural networks (GNNs), an emerging class of machine learning models for graphs, have gained popularity for their superior performance in various graph analytical tasks. Mini-batch training is commonly used to trai…
Graph Neural NetworkRecommendation SystemsDistributed Matrix-Based Sampling for Graph Neural Network Training
Graph Neural Networks (GNNs) offer a compact and computationally efficient way to learn embeddings and classifications on graph data. GNN models are frequently large, making distributed minibatch training necessary. The …
GPUGraph Neural NetworkAccelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units
Graphics Processing Units (GPUs) are commonly used to train and evaluate neural networks efficiently. While previous work in deep learning has focused on accelerating operations on dense matrices/tensors on GPUs, efforts…
GPU