Multi-modal Knowledge Graph Generation with Semantics-enriched Prompts
Multi-modal Knowledge Graphs (MMKGs) have been widely applied across various domains for knowledge representation. However, the existing MMKGs are significantly fewer than required, and their construction faces numerous challenges, particularly in ensuring the selection of high-quality, contextually relevant images for knowledge graph enrichment. To address these challenges, we present a framework for constructing MMKGs from conventional KGs. Furthermore, to generate higher-quality images that are more relevant to the context in the given knowledge graph, we designed a neighbor selection method called Visualizable Structural Neighbor Selection (VSNS). This method consists of two modules: Visualizable Neighbor Selection (VNS) and Structural Neighbor Selection (SNS). The VNS module filters relations that are difficult to visualize, while the SNS module selects neighbors that most effectively capture the structural characteristics of the entity. To evaluate the quality of the generated images, we performed qualitative and quantitative evaluations on two datasets, MKG-Y and DB15K. The experimental results indicate that using the VSNS method to select neighbors results in higher-quality images that are more relevant to the knowledge graph.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph GenerationKnowledge GraphsMulti-modal Knowledge GraphSimilar Papers 제목 키워드 기반
Multimodal RAG for Unstructured Data:Leveraging Modality-Aware Knowledge Graphs with Hybrid Retrieval
Current Retrieval-Augmented Generation (RAG) systems primarily operate on unimodal textual data, limiting their effectiveness on unstructured multimodal documents. Such documents often combine text, images, tables, equat…
Question AnsweringKnowledge GraphsA Pattern to Align Them All: Integrating Different Modalities to Define Multi-Modal Entities
The ability to reason with and integrate different sensory inputs is the foundation underpinning human intelligence and it is the reason for the growing interest in modelling multi-modal information within Knowledge Grap…
AllKnowledge GraphsFedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning
Multimodal graph foundation models aim to learn reusable knowledge from graphs enriched with text, images, attributes, and relational topology, thereby supporting diverse graph-centric and modality-centric tasks. In prac…
Representation LearningSemantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition
Existing vision-based action recognition is susceptible to occlusion and appearance variations, while wearable sensors can alleviate these challenges by capturing human motion with one-dimensional time-series signal. For…
Action RecognitionImage GenerationKnowledge DistillationTime Series+1Fake News Detection via Knowledge-driven Multimodal Graph Convolutional Networks
Nowadays, with the rapid development of social media, there is a great deal of news produced every day. How to detect fake news automatically from a large of multimedia posts has become very important for people, the …
Fake News DetectionNews RecommendationWorld Knowledge