GSINA: Improving Subgraph Extraction for Graph Invariant Learning via Graph Sinkhorn Attention
Graph invariant learning (GIL) has been an effective approach to discovering the invariant relationships between graph data and its labels for different graph learning tasks under various distribution shifts. Many recent endeavors of GIL focus on extracting the invariant subgraph from the input graph for prediction as a regularization strategy to improve the generalization performance of graph learning. Despite their success, such methods also have various limitations in obtaining their invariant subgraphs. In this paper, we provide in-depth analyses of the drawbacks of existing works and propose corresponding principles of our invariant subgraph extraction: 1) the sparsity, to filter out the variant features, 2) the softness, for a broader solution space, and 3) the differentiability, for a soundly end-to-end optimization. To meet these principles in one shot, we leverage the Optimal Transport (OT) theory and propose a novel graph attention mechanism called Graph Sinkhorn Attention (GSINA). This novel approach serves as a powerful regularization method for GIL tasks. By GSINA, we are able to obtain meaningful, differentiable invariant subgraphs with controllable sparsity and softness. Moreover, GSINA is a general graph learning framework that could handle GIL tasks of multiple data grain levels. Extensive experiments on both synthetic and real-world datasets validate the superiority of our GSINA, which outperforms the state-of-the-art GIL methods by large margins on both graph-level tasks and node-level tasks. Our code is publicly available at \url{https://github.com/dingfangyu/GSINA}.
Code (1)
Tasks
Graph AttentionGraph LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Raising the Bar in Graph OOD Generalization: Invariant Learning Beyond Explicit Environment Modeling
Out-of-distribution (OOD) generalization has emerged as a critical challenge in graph learning, as real-world graph data often exhibit diverse and shifting environments that traditional models fail to generalize across. …
Graph LearningSubgraph Aggregation for Out-of-Distribution Generalization on Graphs
Out-of-distribution (OOD) generalization in Graph Neural Networks (GNNs) has gained significant attention due to its critical importance in graph-based predictions in real-world scenarios. Existing methods primarily focu…
Molecular Property PredictionOut-of-Distribution GeneralizationProperty PredictionSubgraph Signal Processing
Graph signal processing, like the graph Fourier transform, requires the full graph signal at every vertex of the graph. However, in practice, only signals at a subset of vertices may be available. We propose a subgraph s…
Graph Invariant Learning with Subgraph Co-mixup for Out-Of-Distribution Generalization
Graph neural networks (GNNs) have been demonstrated to perform well in graph representation learning, but always lacking in generalization capability when tackling out-of-distribution (OOD) data. Graph invariant learning…
Graph Representation LearningOut-of-Distribution GeneralizationRepresentation LearningUnifying Invariant and Variant Features for Graph Out-of-Distribution via Probability of Necessity and Sufficiency
Graph Out-of-Distribution (OOD), requiring that models trained on biased data generalize to the unseen test data, has considerable real-world applications. One of the most mainstream methods is to extract the invariant s…
Graph Learning