Cross-Domain Graph Data Scaling: A Showcase with Diffusion Models
Models for natural language and images benefit from data scaling behavior: the more data fed into the model, the better they perform. This 'better with more' phenomenon enables the effectiveness of large-scale pre-training on vast amounts of data. However, current graph pre-training methods struggle to scale up data due to heterogeneity across graphs. To achieve effective data scaling, we aim to develop a general model that is able to capture diverse data patterns of graphs and can be utilized to adaptively help the downstream tasks. To this end, we propose UniAug, a universal graph structure augmentor built on a diffusion model. We first pre-train a discrete diffusion model on thousands of graphs across domains to learn the graph structural patterns. In the downstream phase, we provide adaptive enhancement by conducting graph structure augmentation with the help of the pre-trained diffusion model via guided generation. By leveraging the pre-trained diffusion model for structure augmentation, we consistently achieve performance improvements across various downstream tasks in a plug-and-play manner. To the best of our knowledge, this study represents the first demonstration of a data-scaling graph structure augmentor on graphs across domains.
Code (2)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Scaling Cross-Domain Content-Based Image Retrieval for E-commerce Snap and Search Application
In this industry talk at ECIR 2022, we illustrate how we approach the main challenges from large scale cross-domain content-based image retrieval using a cascade method and a combination of our visual search and classifi…
Content-Based Image RetrievalImage RetrievalRetrievalAnyGraph: Graph Foundation Model in the Wild
The growing ubiquity of relational data structured as graphs has underscored the need for graph learning models with exceptional generalization capabilities. However, current approaches often struggle to effectively extr…
Graph LearningMixture-of-ExpertsmodelZero-Shot LearningScaling Laws of Decoder-Only Models on the Multilingual Machine Translation Task
Recent studies have showcased remarkable capabilities of decoder-only models in many NLP tasks, including translation. Yet, the machine translation field has been largely dominated by encoder-decoder models based on the …
DecoderMachine TranslationSentenceTranslationLRM-1B: Towards Large Routing Model
Vehicle routing problems (VRPs) are central to combinatorial optimization with significant practical implications. Recent advancements in neural combinatorial optimization (NCO) have demonstrated promising results by lev…
Combinatorial OptimizationmodelTowards Neural Scaling Laws on Graphs
Deep graph models (e.g., graph neural networks and graph transformers) have become important techniques for leveraging knowledge across various types of graphs. Yet, the neural scaling laws on graphs, i.e., how the perfo…
Graph ClassificationLink PredictionNode Classification