Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs
Distilling high-accuracy Graph Neural Networks (GNNs) to low-latency multilayer perceptions (MLPs) on graph tasks has become a hot research topic. However, conventional MLP learning relies almost exclusively on graph nodes and fails to effectively capture the graph structural information. Previous methods address this issue by processing graph edges into extra inputs for MLPs, but such graph structures may be unavailable for various scenarios. To this end, we propose Prototype-Guided Knowledge Distillation (PGKD), which does not require graph edges (edge-free setting) yet learns structure-aware MLPs. Our insight is to distill graph structural information from GNNs. Specifically, we first employ the class prototypes to analyze the impact of graph structures on GNN teachers, and then design two losses to distill such information from GNNs to MLPs. Experimental results on popular graph benchmarks demonstrate the effectiveness and robustness of the proposed PGKD.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Physics-Informed Structure Anchoring With Capture-Aware Prototype Calibration for Cross-Environment RF Fingerprinting
Radio frequency fingerprint identification (RFFI) exploits transmitter-specific hardware imperfections as physicallayer identity cues for Internet of Things (IoT) devices, but deep models often degrade across acquisition…
Representation LearningHunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation
Recent weakly supervised semantic segmentation (WSSS) methods strive to incorporate contextual knowledge to improve the completeness of class activation maps (CAM). In this work, we argue that the knowledge bias between …
Learning TheorySemantic SegmentationWeakly supervised Semantic SegmentationWeakly-Supervised Semantic SegmentationA Graph Foundation Model with Spectral Parsing and Prototype-Guided Spatial Propagation
Graph foundation models aim to learn transferable knowledge from diverse graphs for generalization to unseen graphs and tasks. Unlike text and images, graphs lack a shared vocabulary or regular spatial grid, making cross…
Domain GeneralizationTowards Source-free Domain Adaptive Semantic Segmentation via Importance-aware and Prototype-contrast Learning
Domain adaptive semantic segmentation enables robust pixel-wise understanding in real-world driving scenes. Source-free domain adaptation, as a more practical technique, addresses the concerns of data privacy and storage…
Domain AdaptationSegmentationSemantic SegmentationSource-Free Domain Adaptation+1Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting
Structured radiology reporting promises faster, more consistent communication than free text, but automation remains difficult as models must make many fine-grained, discrete decisions about rare findings and attributes …