DistTGL: Distributed Memory-Based Temporal Graph Neural Network Training

2023-07-14 · Hongkuan Zhou, Da Zheng, Xiang Song, George Karypis, Viktor Prasanna

Memory-based Temporal Graph Neural Networks are powerful tools in dynamic graph representation learning and have demonstrated superior performance in many real-world applications. However, their node memory favors smaller batch sizes to capture more dependencies in graph events and needs to be maintained synchronously across all trainers. As a result, existing frameworks suffer from accuracy loss when scaling to multiple GPUs. Evenworse, the tremendous overhead to synchronize the node memory make it impractical to be deployed to distributed GPU clusters. In this work, we propose DistTGL -- an efficient and scalable solution to train memory-based TGNNs on distributed GPU clusters. DistTGL has three improvements over existing solutions: an enhanced TGNN model, a novel training algorithm, and an optimized system. In experiments, DistTGL achieves near-linear convergence speedup, outperforming state-of-the-art single-machine method by 14.5% in accuracy and 10.17x in training throughput.

📄 PDF Abstract BibTeX arXiv:2307.07649

Code (0)

등록된 구현이 없습니다.

Tasks

GPUGraph Neural Network Graph Representation Learning Representation Learning

PGT-I: Scaling Spatiotemporal GNNs with Memory-Efficient Distributed Training

2025-07-15 · Seth Ockerman, Amal Gueroudji, Tanwi Mallick, Yixuan He 외

Spatiotemporal graph neural networks (ST-GNNs) are powerful tools for modeling spatial and temporal data dependencies. However, their applications have been limited primarily to small-scale datasets because of memory con…

graph partitioning

GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs

2023-11-29 · Yuchen Zhong, Guangming Sheng, Tianzuo Qin, Minjie Wang 외

Graph Neural Networks (GNNs) play a crucial role in various fields. However, most existing deep graph learning frameworks assume pre-stored static graphs and do not support training on graph streams. In contrast, many re…

CPU GPU Graph Learning Graph Representation Learning+2

VL-MemKnG: Hybrid Memory with a Spatio-Temporal Knowledge Graph for Question Answering over Long Egocentric Navigation Trajectories

2026-06-15 · Svetlana Lukina, Mohamad Al Mdfaa, Gloria Haro, Sergey Zagoruyko 외 arxiv

Answering navigation-relevant questions over long egocentric videos requires retrieving and organizing evidence distributed across distant temporal moments while maintaining spatial and contextual consistency. Although l…

Video Question Answering Knowledge Graphs

Learning Successor Features with Distributed Hebbian Temporal Memory

2023-10-20 · Evgenii Dzhivelikian, Petr Kuderov, Aleksandr I. Panov

This paper presents a novel approach to address the challenge of online temporal memory learning for decision-making under uncertainty in non-stationary, partially observable environments. The proposed algorithm, Distrib…

Decision Making Decision Making Under Uncertainty Representation Learning

CATGNN: Cost-Efficient and Scalable Distributed Training for Graph Neural Networks

2024-04-02 · Xin Huang, Weipeng Zhuo, Minh Phu Vuong, Shiju Li 외

Graph neural networks have been shown successful in recent years. While different GNN architectures and training systems have been developed, GNN training on large-scale real-world graphs still remains challenging. Exist…