Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces
Continual graph learning routinely finds its role in a variety of real-world applications where the graph data with different tasks come sequentially. Despite the success of prior works, it still faces great challenges. On the one hand, existing methods work with the zero-curvature Euclidean space, and largely ignore the fact that curvature varies over the coming graph sequence. On the other hand, continual learners in the literature rely on abundant labels, but labeling graph in practice is particularly hard especially for the continuously emerging graphs on-the-fly. To address the aforementioned challenges, we propose to explore a challenging yet practical problem, the self-supervised continual graph learning in adaptive Riemannian spaces. In this paper, we propose a novel self-supervised Riemannian Graph Continual Learner (RieGrace). In RieGrace, we first design an Adaptive Riemannian GCN (AdaRGCN), a unified GCN coupled with a neural curvature adapter, so that Riemannian space is shaped by the learnt curvature adaptive to each graph. Then, we present a Label-free Lorentz Distillation approach, in which we create teacher-student AdaRGCN for the graph sequence. The student successively performs intra-distillation from itself and inter-distillation from the teacher so as to consolidate knowledge without catastrophic forgetting. In particular, we propose a theoretically grounded Generalized Lorentz Projection for the contrastive distillation in Riemannian space. Extensive experiments on the benchmark datasets show the superiority of RieGrace, and additionally, we investigate on how curvature changes over the graph sequence.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Self-supervised Mixed-curvature Graph Neural Network
Graph representation learning received increasing attentions in recent years. Most of existing methods ignore the complexity of the graph structures and restrict graphs in a single constant-curvature representation space…
Contrastive LearningGraph Neural NetworkGraph Representation LearningRepresentation LearningA Self-supervised Riemannian GNN with Time Varying Curvature for Temporal Graph Learning
Representation learning on temporal graphs has drawn considerable research attention owing to its fundamental importance in a wide spectrum of real-world applications. Though a number of studies succeed in obtaining time…
Graph LearningGraph Neural NetworkGraph Representation LearningRepresentation Learning+1DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing
Graph Neural Networks (GNNs) have shown great power for learning and mining on graphs, and Graph Structure Learning (GSL) plays an important role in boosting GNNs with a refined graph. In the literature, most GSL solutio…
Contrastive LearningGraph structure learningNode ClassificationPR-CapsNet: Pseudo-Riemannian Capsule Network with Adaptive Curvature Routing for Graph Learning
Capsule Networks (CapsNets) show exceptional graph representation capacity via dynamic routing and vectorized hierarchical representations, but they model the complex geometries of real\-world graphs poorly by fixed\-cur…
Graph Representation LearningGraph ClassificationGraph LearningMotif-aware Riemannian Graph Neural Network with Generative-Contrastive Learning
Graphs are typical non-Euclidean data of complex structures. In recent years, Riemannian graph representation learning has emerged as an exciting alternative to Euclidean ones. However, Riemannian methods are still in an…
Contrastive LearningGraph Neural NetworkGraph Representation LearningRepresentation Learning