Multimodal Analogical Reasoning over Knowledge Graphs
Analogical reasoning is fundamental to human cognition and holds an important place in various fields. However, previous studies mainly focus on single-modal analogical reasoning and ignore taking advantage of structure knowledge. Notably, the research in cognitive psychology has demonstrated that information from multimodal sources always brings more powerful cognitive transfer than single modality sources. To this end, we introduce the new task of multimodal analogical reasoning over knowledge graphs, which requires multimodal reasoning ability with the help of background knowledge. Specifically, we construct a Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained Transformer baselines, illustrating the potential challenges of the proposed task. We further propose a novel model-agnostic Multimodal analogical reasoning framework with Transformer (MarT) motivated by the structure mapping theory, which can obtain better performance. Code and datasets are available in https://github.com/zjunlp/MKG_Analogy.
Code (2)
Tasks
Graph EmbeddingKnowledge Graph EmbeddingKnowledge GraphsMultimodal ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing multimodal analogical reasoning with Logic Augmented Generation
Recent advances in Large Language Models have demonstrated their capabilities across a variety of tasks. However, automatically extracting implicit knowledge from natural language remains a significant challenge, as mach…
Knowledge GraphsText GenerationANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base
Analogical reasoning is a fundamental cognitive ability of humans. However, current language models (LMs) still struggle to achieve human-like performance in analogical reasoning tasks due to a lack of resources for mode…
Knowledge GraphsCan Multimodal Large Language Model Think Analogically?
Analogical reasoning, particularly in multimodal contexts, is the foundation of human perception and creativity. Multimodal Large Language Model (MLLM) has recently sparked considerable discussion due to its emergent cap…
Language ModelingLanguage ModellingLarge Language Modelmodel+1KGPrune: a Web Application to Extract Subgraphs of Interest from Wikidata with Analogical Pruning
Knowledge graphs (KGs) have become ubiquitous publicly available knowledge sources, and are nowadays covering an ever increasing array of domains. However, not all knowledge represented is useful or pertaining when consi…
Knowledge GraphsAnalogical Inference Enhanced Knowledge Graph Embedding
Knowledge graph embedding (KGE), which maps entities and relations in a knowledge graph into continuous vector spaces, has achieved great success in predicting missing links in knowledge graphs. However, knowledge graphs…
Graph EmbeddingKnowledge Graph EmbeddingKnowledge GraphsLink Prediction