Data-Free Adversarial Knowledge Distillation for Graph Neural Networks
Graph neural networks (GNNs) have been widely used in modeling graph structured data, owing to its impressive performance in a wide range of practical applications. Recently, knowledge distillation (KD) for GNNs has enabled remarkable progress in graph model compression and knowledge transfer. However, most of the existing KD methods require a large volume of real data, which are not readily available in practice, and may preclude their applicability in scenarios where the teacher model is trained on rare or hard to acquire datasets. To address this problem, we propose the first end-to-end framework for data-free adversarial knowledge distillation on graph structured data (DFAD-GNN). To be specific, our DFAD-GNN employs a generative adversarial network, which mainly consists of three components: a pre-trained teacher model and a student model are regarded as two discriminators, and a generator is utilized for deriving training graphs to distill knowledge from the teacher model into the student model. Extensive experiments on various benchmark models and six representative datasets demonstrate that our DFAD-GNN significantly surpasses state-of-the-art data-free baselines in the graph classification task.
Code (0)
등록된 구현이 없습니다.
Tasks
Generative Adversarial NetworkGraph ClassificationKnowledge DistillationModel CompressionTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adversarial Curriculum Graph-Free Knowledge Distillation for Graph Neural Networks
Data-free Knowledge Distillation (DFKD) is a method that constructs pseudo-samples using a generator without real data, and transfers knowledge from a teacher model to a student by enforcing the student to overcome dimen…
Data-free Knowledge DistillationKnowledge DistillationEnhancing Data-Free Adversarial Distillation with Activation Regularization and Virtual Interpolation
Knowledge distillation refers to a technique of transferring the knowledge from a large learned model or an ensemble of learned models to a small model. This method relies on access to the original training set, which mi…
Knowledge DistillationAdversarial Self-Supervised Data-Free Distillation for Text Classification
Large pre-trained transformer-based language models have achieved impressive results on a wide range of NLP tasks. In the past few years, Knowledge Distillation(KD) has become a popular paradigm to compress a computation…
ClassificationGeneral ClassificationKnowledge Distillationtext-classification+2Online Adversarial Knowledge Distillation for Graph Neural Networks
Knowledge distillation, a technique recently gaining popularity for enhancing model generalization in Convolutional Neural Networks (CNNs), operates under the assumption that both teacher and student models are trained o…
Knowledge DistillationFedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks
While existing federated learning approaches primarily focus on aggregating local models to construct a global model, in realistic settings, some clients may be reluctant to share their private models due to the inclusio…
Data-free Knowledge DistillationFederated LearningGenerative Adversarial NetworkKnowledge Distillation