Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization
Graph neural networks (GNNs) have achieved superior performance in various applications, but training dedicated GNNs can be costly for large-scale graphs. Some recent work started to study the pre-training of GNNs. However, none of them provide theoretical insights into the design of their frameworks, or clear requirements and guarantees towards their transferability. In this work, we establish a theoretically grounded and practically useful framework for the transfer learning of GNNs. Firstly, we propose a novel view towards the essential graph information and advocate the capturing of it as the goal of transferable GNN training, which motivates the design of EGI (Ego-Graph Information maximization) to analytically achieve this goal. Secondly, when node features are structure-relevant, we conduct an analysis of EGI transferability regarding the difference between the local graph Laplacians of the source and target graphs. We conduct controlled synthetic experiments to directly justify our theoretical conclusions. Comprehensive experiments on two real-world network datasets show consistent results in the analyzed setting of direct-transfering, while those on large-scale knowledge graphs show promising results in the more practical setting of transfering with fine-tuning.
Code (1)
Tasks
Knowledge GraphsTransfer LearningSimilar Papers 제목 키워드 기반
Deep Graph Clustering via Mutual Information Maximization and Mixture Model
Attributed graph clustering or community detection which learns to cluster the nodes of a graph is a challenging task in graph analysis. In this paper, we introduce a contrastive learning framework for learning clusterin…
ClusteringCommunity DetectionContrastive LearningGraph Clustering+1Revisiting Modularity Maximization for Graph Clustering: A Contrastive Learning Perspective
Graph clustering, a fundamental and challenging task in graph mining, aims to classify nodes in a graph into several disjoint clusters. In recent years, graph contrastive learning (GCL) has emerged as a dominant line of …
ClusteringCommunity DetectionContrastive LearningGraph Clustering+1G5: A Universal GRAPH-BERT for Graph-to-Graph Transfer and Apocalypse Learning
The recent GRAPH-BERT model introduces a new approach to learning graph representations merely based on the attention mechanism. GRAPH-BERT provides an opportunity for transferring pre-trained models and learned graph re…
Classification ConsistencyGraph Representation LearningRepresentation LearningAn Expectation Conditional Maximization approach for Gaussian graphical models
Bayesian graphical models are a useful tool for understanding dependence relationships among many variables, particularly in situations with external prior information. In high-dimensional settings, the space of possible…
Variable SelectionAssociation Graph Learning for Multi-Task Classification with Category Shifts
In this paper, we focus on multi-task classification, where related classification tasks share the same label space and are learned simultaneously. In particular, we tackle a new setting, which is more realistic than cur…
General ClassificationGraph LearningLesion ClassificationSkin Lesion Classification+1