TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution

2026-03-29 · Yinlin Zhu, Xunkai Li, Di Wu, Wang Luo, Miao Hu, Di Wu arxiv

Multimodal-attributed graphs (MAGs) are a fundamental data structure for multimodal graph learning (MGL), enabling both graph-centric and modality-centric tasks. However, our empirical analysis reveals inherent topology quality limitations in real-world MAGs, including noisy interactions, missing connections, and task-agnostic relational structures. A single graph derived from generic relationships is therefore unlikely to be universally optimal for diverse downstream tasks. To address this challenge, we propose Task-aware Modality and Topology co-Evolution (TMTE), a novel MGL framework that jointly and iteratively optimizes graph topology and multimodal representations toward the target task. TMTE is motivated by the bidirectional coupling between modality and topology: multimodal attributes induce relational structures, while graph topology shapes modality representations. Concretely, TMTE casts topology evolution as multi-perspective metric learning over modality embeddings with an anchor-based approximation, and formulates modality evolution as smoothness-regularized fusion with cross-modal alignment, yielding a closed-loop task-aware co-evolution process. Extensive experiments on 9 MAG datasets and 1 non-graph multimodal dataset across 6 graph-centric and modality-centric tasks show that TMTE consistently achieves state-of-the-art performance. Our code is available at https://anonymous.4open.science/r/TMTE-1873.

📄 PDF Abstract BibTeX arXiv:2603.27723

Code (0)

등록된 구현이 없습니다.

Tasks

Metric Learning Graph Learning

Language Models Enable Data-Augmented Synthesis Planning for Inorganic Materials

2025-06-14 · Thorben Prein, Elton Pan, Janik Jehkul, Steffen Weinmann 외

Inorganic synthesis planning currently relies primarily on heuristic approaches or machine-learning models trained on limited datasets, which constrains its generality. We demonstrate that language models, without task-s…

Multimodal RAG for Unstructured Data:Leveraging Modality-Aware Knowledge Graphs with Hybrid Retrieval

2025-10-16 · Rashmi R, Vidyadhar Upadhya arxiv

Current Retrieval-Augmented Generation (RAG) systems primarily operate on unimodal textual data, limiting their effectiveness on unstructured multimodal documents. Such documents often combine text, images, tables, equat…

Question Answering Knowledge Graphs

Stable Multimodal Graph Unlearning via Feature-Dimension Aware Quantile Selection

2026-05-05 · Jingjing Zhou, Yongshuai Yang, Qing Qing, Ziqi Xu 외 arxiv

Graph unlearning remains a critical technique for supporting privacy-preserving and sustainable multimodal graph learning. However, we observe that existing unlearning strategies tend to apply uniform parameter selection…

Graph Neural Network Graph Learning

DiffusionCom: Structure-Aware Multimodal Diffusion Model for Multimodal Knowledge Graph Completion

2025-04-09 · Wei Huang, Meiyu Liang, Peining Li, Xu Hou 외

Most current MKGC approaches are predominantly based on discriminative models that maximize conditional likelihood. These approaches struggle to efficiently capture the complex connections in real-world knowledge graphs,…

Graph Attention Knowledge Graph Completion Knowledge Graphs Representation Learning+1

Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework

2026-07-17 · Xunkai Li, Guohao Fu, Yuming Ai, Zhengyu Wu 외 arxiv

Multimodal-attributed graphs (MAGs), whose nodes carry modalities such as images and text alongside topological structure, now pervade applications including social platforms, e-commerce, and biomedical networks, offerin…

Federated Learning Few-Shot Learning Graph Learning