Overcoming Catastrophic Forgetting in Graph Neural Networks with Experience Replay
Graph Neural Networks (GNNs) have recently received significant research attention due to their superior performance on a variety of graph-related learning tasks. Most of the current works focus on either static or dynamic graph settings, addressing a single particular task, e.g., node/graph classification, link prediction. In this work, we investigate the question: can GNNs be applied to continuously learning a sequence of tasks? Towards that, we explore the Continual Graph Learning (CGL) paradigm and present the Experience Replay based framework ER-GNN for CGL to alleviate the catastrophic forgetting problem in existing GNNs. ER-GNN stores knowledge from previous tasks as experiences and replays them when learning new tasks to mitigate the catastrophic forgetting issue. We propose three experience node selection strategies: mean of feature, coverage maximization, and influence maximization, to guide the process of selecting experience nodes. Extensive experiments on three benchmark datasets demonstrate the effectiveness of our ER-GNN and shed light on the incremental graph (non-Euclidean) structure learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph ClassificationGraph LearningLink PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay
Despite huge success, deep networks are unable to learn effectively in sequential multitask learning settings as they forget the past learned tasks after learning new tasks. Inspired from complementary learning systems t…
Distilled Replay: Overcoming Forgetting through Synthetic Samples
Replay strategies are Continual Learning techniques which mitigate catastrophic forgetting by keeping a buffer of patterns from previous experiences, which are interleaved with new data during training. The amount of pat…
Continual LearningHistory Repeats: Overcoming Catastrophic Forgetting For Event-Centric Temporal Knowledge Graph Completion
Temporal knowledge graph (TKG) completion models typically rely on having access to the entire graph during training. However, in real-world scenarios, TKG data is often received incrementally as events unfold, leading t…
ClusteringKnowledge Graph CompletionTemporal Knowledge Graph CompletionContinual Learning of Personalized Generative Face Models with Experience Replay
We introduce a novel continual learning problem: how to sequentially update the weights of a personalized 2D and 3D generative face model as new batches of photos in different appearances, styles, poses, and lighting are…
Continual LearningFace ModelSelective Experience Replay for Lifelong Learning
Deep reinforcement learning has emerged as a powerful tool for a variety of learning tasks, however deep nets typically exhibit forgetting when learning multiple tasks in sequence. To mitigate forgetting, we propose an e…
Deep Reinforcement LearningLifelong learningReinforcement Learning