Characterizing and Understanding HGNN Training on GPUs
Owing to their remarkable representation capabilities for heterogeneous graph data, Heterogeneous Graph Neural Networks (HGNNs) have been widely adopted in many critical real-world domains such as recommendation systems and medical analysis. Prior to their practical application, identifying the optimal HGNN model parameters tailored to specific tasks through extensive training is a time-consuming and costly process. To enhance the efficiency of HGNN training, it is essential to characterize and analyze the execution semantics and patterns within the training process to identify performance bottlenecks. In this study, we conduct an in-depth quantification and analysis of two mainstream HGNN training scenarios, including single-GPU and multi-GPU distributed training. Based on the characterization results, we disclose the performance bottlenecks and their underlying causes in different HGNN training scenarios and provide optimization guidelines from both software and hardware perspectives.
Code (0)
등록된 구현이 없습니다.
Tasks
GPURecommendation SystemsSimilar Papers 제목 키워드 기반
Characterizing and Understanding Distributed GNN Training on GPUs
Graph neural network (GNN) has been demonstrated to be a powerful model in many domains for its effectiveness in learning over graphs. To scale GNN training for large graphs, a widely adopted approach is distributed trai…
Graph Neural NetworkSiHGNN: Leveraging Properties of Semantic Graphs for Efficient HGNN Acceleration
Heterogeneous Graph Neural Networks (HGNNs) have expanded graph representation learning to heterogeneous graph fields. Recent studies have demonstrated their superior performance across various applications, including me…
Graph GenerationGraph Representation LearningRecommendation SystemsRepresentation Learning+1Seg-HGNN: Unsupervised and Light-Weight Image Segmentation with Hyperbolic Graph Neural Networks
Image analysis in the euclidean space through linear hyperspaces is well studied. However, in the quest for more effective image representations, we turn to hyperbolic manifolds. They provide a compelling alternative to …
Graph Neural NetworkImage SegmentationSemantic SegmentationAre we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements. In this work, …
BenchmarkingHeterogeneous Node ClassificationLHGNN: Local-Higher Order Graph Neural Networks For Audio Classification and Tagging
Transformers have set new benchmarks in audio processing tasks, leveraging self-attention mechanisms to capture complex patterns and dependencies within audio data. However, their focus on pairwise interactions limits th…
Audio ClassificationGraph Neural Network