Efficiently Forgetting What You Have Learned in Graph Representation Learning via Projection
As privacy protection receives much attention, unlearning the effect of a specific node from a pre-trained graph learning model has become equally important. However, due to the node dependency in the graph-structured data, representation unlearning in Graph Neural Networks (GNNs) is challenging and less well explored. In this paper, we fill in this gap by first studying the unlearning problem in linear-GNNs, and then introducing its extension to non-linear structures. Given a set of nodes to unlearn, we propose PROJECTOR that unlearns by projecting the weight parameters of the pre-trained model onto a subspace that is irrelevant to features of the nodes to be forgotten. PROJECTOR could overcome the challenges caused by node dependency and enjoys a perfect data removal, i.e., the unlearned model parameters do not contain any information about the unlearned node features which is guaranteed by algorithmic construction. Empirical results on real-world datasets illustrate the effectiveness and efficiency of PROJECTOR.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph LearningGraph Representation LearningRepresentation LearningSimilar Papers 제목 키워드 기반
Continual Neural Topic Model
In continual learning, our aim is to learn a new task without forgetting what was learned previously. In topic models, this translates to learning new topic models without forgetting previously learned topics. Previous w…
Continual LearningTopic ModelsTowards Aligned Data Forgetting via Twin Machine Unlearning
Modern privacy regulations have spurred the evolution of machine unlearning, a technique enabling a trained model to efficiently forget specific training data. In prior unlearning methods, the concept of "data forgetting…
Machine UnlearningSupportNet: solving catastrophic forgetting in class incremental learning with support data
A plain well-trained deep learning model often does not have the ability to learn new knowledge without forgetting the previously learned knowledge, which is known as catastrophic forgetting. Here we propose a novel meth…
class-incremental learningClass Incremental LearningDeep LearningIncremental LearningBrain-inspired replay for continual learning with artificial neural networks
Artificial neural networks suffer from catastrophic forgetting. Unlike humans, when these networks are trained on something new, they rapidly forget what was learned before. In the brain, a mechanism thought to be import…
class-incremental learningClass Incremental LearningContinual LearningIncremental LearningVariational Prototype Replays for Continual Learning
Continual learning refers to the ability to acquire and transfer knowledge without catastrophically forgetting what was previously learned. In this work, we consider \emph{few-shot} continual learning in classification t…
Continual LearningGeneral ClassificationMetric LearningNovel Concepts