GraphProp: Training the Graph Foundation Models using Graph Properties
This work focuses on training graph foundation models (GFMs) that have strong generalization ability in graph-level tasks such as graph classification. Effective GFM training requires capturing information consistent across different domains. We discover that graph structures provide more consistent cross-domain information compared to node features and graph labels. However, traditional GFMs primarily focus on transferring node features from various domains into a unified representation space but often lack structural cross-domain generalization. To address this, we introduce GraphProp, which emphasizes structural generalization. The training process of GraphProp consists of two main phases. First, we train a structural GFM by predicting graph invariants. Since graph invariants are properties of graphs that depend only on the abstract structure, not on particular labellings or drawings of the graph, this structural GFM has a strong ability to capture the abstract structural information and provide discriminative graph representations comparable across diverse domains. In the second phase, we use the representations given by the structural GFM as positional encodings to train a comprehensive GFM. This phase utilizes domain-specific node attributes and graph labels to further improve cross-domain node feature generalization. Our experiments demonstrate that GraphProp significantly outperforms the competitors in supervised learning and few-shot learning, especially in handling graphs without node attributes.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain GeneralizationGraph ClassificationFew-Shot LearningSimilar Papers 제목 키워드 기반
Beyond Low Rank: A Graph-Based Propagation Approach to Tensor Completion for Multi-Acquisition Scenarios
Tensor completion refers to the problem of recovering the missing, corrupted or unobserved entries in data represented by tensors. In this paper, we tackle the tensor completion problem in the scenario in which multiple …
OpenGraph: Towards Open Graph Foundation Models
Graph learning has become essential in various domains, including recommendation systems and social network analysis. Graph Neural Networks (GNNs) have emerged as promising techniques for encoding structural information …
Data AugmentationGraph LearningLanguage ModelingLanguage Modelling+4GOFA: A Generative One-For-All Model for Joint Graph Language Modeling
Foundation models, such as Large Language Models (LLMs) or Large Vision Models (LVMs), have emerged as one of the most powerful tools in the respective fields. However, unlike text and image data, graph data do not have …
AllLanguage ModelingLanguage ModellingQuestion AnsweringTowards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees
Foundation models are pretrained on large-scale corpora to learn generalizable patterns across domains and tasks -- such as contours, textures, and edges in images, or tokens and sentences in text. In contrast, discoveri…
Graph Neural NetworkIn-Context LearningZero-shot GeneralizationZero-Shot LearningCan TabPFN Compete with GNNs for Node Classification via Graph Tabularization?
Foundation models pretrained on large data have demonstrated remarkable zero-shot generalization capabilities across domains. Building on the success of TabPFN for tabular data and its recent extension to time series, we…
Zero-shot GeneralizationFeature EngineeringNode Classification