홈 › Papers

DGCL: an efficient communication library for distributed GNN training

2021-04-21 · Proceedings of the Sixteenth European Conference on Computer Systems 2021 4 · Zhenkun Cai, Xiao Yan, Yidi Wu, Kaihao Ma, James Cheng, Fan Yu

Graph neural networks (GNNs) have gained increasing popularity in many areas such as e-commerce, social networks and bio-informatics. Distributed GNN training is essential for handling large graphs and reducing the execution time. However, for distributed GNN training, a peer-to-peer communication strategy suffers from high communication overheads. Also, different GPUs require different remote vertex embeddings, which leads to an irregular communication pattern and renders existing communication planning solutions unsuitable. We propose the distributed graph communication library (DGCL) for efficient GNN training on multiple GPUs. At the heart of DGCL is a communication planning algorithm tailored for GNN training, which jointly considers fully utilizing fast links, fusing communication, avoiding contention and balancing loads on different links. DGCL can be easily adopted to extend existing single-GPU GNN systems to distributed training. We conducted extensive experiments on different datasets and network configurations to compare DGCL with alternative communication schemes. In our experiments, DGCL reduces the communication time of the peer-to-peer communication by 77.5% on average and the training time for an epoch by up to 47%.

📄 PDF Abstract BibTeX

Code (1)

czkkkkkk/gccl

Tasks

GPU

Domain Generalizable Continual Learning

2025-10-19 · Hongwei Yan, Guanglong Sun, Zhiqi Kang, Yi Zhong 외 arxiv

To adapt effectively to dynamic real-world environments, intelligent systems must continually acquire new skills while generalizing them to diverse, unseen scenarios. Here, we introduce a novel and realistic setting name…

Representation Learning Continual Learning

Horovod: fast and easy distributed deep learning in TensorFlow

2018-02-15 · Alexander Sergeev, Mike Del Balso

Training modern deep learning models requires large amounts of computation, often provided by GPUs. Scaling computation from one GPU to many can enable much faster training and research progress but entails two complicat…

Deep Learning GPU

FairDgcl: Fairness-aware Recommendation with Dynamic Graph Contrastive Learning

2024-10-23 · Wei Chen, Meng Yuan, Zhao Zhang, Ruobing Xie 외

As trustworthy AI continues to advance, the fairness issue in recommendations has received increasing attention. A recommender system is considered unfair when it produces unequal outcomes for different user groups based…

Contrastive Learning Data Augmentation Fairness Recommendation Systems

Communication Optimization for Distributed Training: Architecture, Advances, and Opportunities

2024-03-12 · Yunze Wei, Tianshuo Hu, Cong Liang, Yong Cui

The past few years have witnessed the flourishing of large-scale deep neural network models with ever-growing parameter numbers. Training such large-scale models typically requires massive memory and computing resources,…

GPU

Squeeze and Excitation: A Weighted Graph Contrastive Learning for Collaborative Filtering

2025-04-06 · Zheyu Chen, Jinfeng Xu, Yutong Wei, Ziyue Peng

Contrastive Learning (CL) has recently emerged as a powerful technique in recommendation systems, particularly for its capability to harness self-supervised signals from perturbed views to mitigate the persistent challen…

Collaborative Filtering Contrastive Learning Recommendation Systems Representation Learning