Fine-Tuning Graph Neural Networks via Graph Topology induced Optimal Transport

2022-03-20 · Jiying Zhang, Xi Xiao, Long-Kai Huang, Yu Rong, Yatao Bian

Recently, the pretrain-finetuning paradigm has attracted tons of attention in graph learning community due to its power of alleviating the lack of labels problem in many real-world applications. Current studies use existing techniques, such as weight constraint, representation constraint, which are derived from images or text data, to transfer the invariant knowledge from the pre-train stage to fine-tuning stage. However, these methods failed to preserve invariances from graph structure and Graph Neural Network (GNN) style models. In this paper, we present a novel optimal transport-based fine-tuning framework called GTOT-Tuning, namely, Graph Topology induced Optimal Transport fine-Tuning, for GNN style backbones. GTOT-Tuning is required to utilize the property of graph data to enhance the preservation of representation produced by fine-tuned networks. Toward this goal, we formulate graph local knowledge transfer as an Optimal Transport (OT) problem with a structural prior and construct the GTOT regularizer to constrain the fine-tuned model behaviors. By using the adjacency relationship amongst nodes, the GTOT regularizer achieves node-level optimal transport procedures and reduces redundant transport procedures, resulting in efficient knowledge transfer from the pre-trained models. We evaluate GTOT-Tuning on eight downstream tasks with various GNN backbones and demonstrate that it achieves state-of-the-art fine-tuning performance for GNNs.

📄 PDF Abstract BibTeX arXiv:2203.10453

Code (1)

youjibiying/gtot-tuning 공식 구현 pytorch

Tasks

Graph Classification Graph Learning Graph Neural Network Molecular Property Prediction Transfer Learning

Methods 이 논문이 사용한 방법론

Graph Neural Network 설명 없음

Network Games Induced Prior for Graph Topology Learning

2024-10-31 · Chenyue Zhang, Shangyuan Liu, Hoi-To Wai, Anthony Man-Cho So

Learning the graph topology of a complex network is challenging due to limited data availability and imprecise data models. A common remedy in existing works is to incorporate priors such as sparsity or modularity which …

Graph Learning

Stabilizing Decentralized Federated Fine-Tuning via Topology-Aware Alternating LoRA

2026-01-31 · Xiaoyu Wang, Xiaotian Li, Zhixiang Zhou, Chen Li 외 arxiv

Decentralized federated learning (DFL), a serverless variant of federated learning, poses unique challenges for parameter-efficient fine-tuning due to the factorized structure of low-rank adaptation (LoRA). Unlike linear…

parameter-efficient fine-tuning Federated Learning

Defense-as-a-Service: Black-box Shielding against Backdoored Graph Models

2024-10-07 · Xiao Yang, Kai Zhou, Yuni Lai, Gaolei Li

With the trend of large graph learning models, business owners tend to employ a model provided by a third party to deliver business services to users. However, these models might be backdoored, and malicious users can su…

Backdoor Attack Clustering Graph Classification Graph Learning

SMGFM: Spectral Multimodal Graph Pretraining for Multimodal-Attributed Graphs

2026-06-11 · Zhengyu Wu, Xu Wang, Hongchao Qin, Xunkai Li 외 arxiv

Multimodal-attributed graphs (MAGs) couple graph topology with node semantics from text, images, and other modalities. Traditional graph learning contextualizes node semantics by coupling topology with node features. How…

Graph Learning

Robust Graph Fine-Tuning with Adversarial Graph Prompting

2026-01-01 · Ziyan Zhang, Bo Jiang, Jin Tang arxiv

Parameter-Efficient Fine-Tuning (PEFT) method has emerged as a dominant paradigm for adapting pre-trained GNN models to downstream tasks. However, existing PEFT methods usually exhibit significant vulnerability to variou…

parameter-efficient fine-tuning