GSTAM: Efficient Graph Distillation with Structural Attention-Matching

2024-08-29 · Arash Rasti-Meymandi, Ahmad Sajedi, Zhaopan Xu, Konstantinos N. Plataniotis

Graph distillation has emerged as a solution for reducing large graph datasets to smaller, more manageable, and informative ones. Existing methods primarily target node classification, involve computationally intensive processes, and fail to capture the true distribution of the full graph dataset. To address these issues, we introduce Graph Distillation with Structural Attention Matching (GSTAM), a novel method for condensing graph classification datasets. GSTAM leverages the attention maps of GNNs to distill structural information from the original dataset into synthetic graphs. The structural attention-matching mechanism exploits the areas of the input graph that GNNs prioritize for classification, effectively distilling such information into the synthetic graphs and improving overall distillation performance. Comprehensive experiments demonstrate GSTAM's superiority over existing methods, achieving 0.45% to 6.5% better performance in extreme condensation ratios, highlighting its potential use in advancing distillation for graph classification tasks (Code available at https://github.com/arashrasti96/GSTAM).

📄 PDF Abstract BibTeX arXiv:2408.16871

Code (1)

arashrasti96/gstam 공식 구현 pytorch

Tasks

Classification Graph Classification Node Classification

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

DSBD: Dual-Aligned Structural Basis Distillation for Graph Domain Adaptation

2026-04-03 · Yingxu Wang, Kunyu Zhang, Jiaxin Huang, Mengzhu Wang 외 arxiv

Graph domain adaptation (GDA) aims to transfer knowledge from a labeled source graph to an unlabeled target graph under distribution shifts. However, existing methods are largely feature-centric and overlook structural d…

GRAPH DOMAIN ADAPTATION

LayoutGMN: Neural Graph Matching for Structural Layout Similarity

2020-12-11 · CVPR 2021 1 · Akshay Gadi Patil, Manyi Li, Matthew Fisher, Manolis Savva 외

We present a deep neural network to predict structural similarity between 2D layouts by leveraging Graph Matching Networks (GMN). Our network, coined LayoutGMN, learns the layout metric via neural graph matching, using a…

Graph Matching Metric Learning Retrieval Triplet

Exploring Graph-based Knowledge: Multi-Level Feature Distillation via Channels Relational Graph

2024-05-14 · Zhiwei Wang, Jun Huang, Longhua Ma, Chengyu Wu 외

In visual tasks, large teacher models capture essential features and deep information, enhancing performance. However, distilling this information into smaller student models often leads to performance loss due to struct…

Multiresolution Graph Attention Networks for Relevance Matching

2019-02-27 · Ting Zhang, Bang Liu, Di Niu, Kunfeng Lai 외

A large number of deep learning models have been proposed for the text matching problem, which is at the core of various typical natural language processing (NLP) tasks. However, existing deep models are mainly designed …

Graph Attention Information Retrieval Paraphrase Identification Question Answering+2

Propagate & Distill: Towards Effective Graph Learners Using Propagation-Embracing MLPs

2023-11-29 · Yong-Min Shin, Won-Yong Shin

Recent studies attempted to utilize multilayer perceptrons (MLPs) to solve semisupervised node classification on graphs, by training a student MLP by knowledge distillation from a teacher graph neural network (GNN). Whil…

Graph Neural Network Knowledge Distillation Node Classification