paper-with-me

홈 › Papers

MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

2025-03-24 · Shuo Yang, Siwen Luo, Soyeon Caren Han, Eduard Hovy

Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limiting their robustness in real-world scenarios. To address this, we introduce MAGIC-VQA, a novel framework that enhances VQA by systematically integrating commonsense knowledge with LVLMs. MAGIC-VQA employs a three-stage process: (1) Explicit Knowledge Integration from external sources, (2) By-Type Post-Processing for contextual refinement, and (3) Implicit Knowledge Augmentation using a Graph Neural Network (GNN) for structured reasoning. While GNNs bring greater depth to structured inference, they enable superior relational inference beyond LVLMs. MAGIC-VQA bridges a key gap by unifying commonsensse knowledge with LVLM-driven reasoning, eliminating the need for extensive pre-training or complex prompt tuning. Our framework achieves state-of-the-art performance on benchmark datasets, significantly improving commonsense reasoning in VQA.

📄 PDF Abstract BibTeX arXiv:2503.18491

Code (0)

등록된 구현이 없습니다.

Tasks

Graph Neural NetworkQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)

Methods 이 논문이 사용한 방법론

Graph Neural Network 설명 없음

Similar Papers 제목 키워드 기반

Visual Commonsense in Pretrained Unimodal and Multimodal Models

2022-01-16 · ACL ARR January 2022 1 · Anonymous

Our commonsense knowledge about objects includes their typical visual attributes; we know that bananas are typically yellow or green, and not purple. Text and image corpora, being subject to reporting bias, represent thi…

AttributeWorld Knowledge

MMCOMET: A Large-Scale Multimodal Commonsense Knowledge Graph for Contextual Reasoning

2026-03-01 · Eileen Wang, Hiba Arnaout, Dhita Pratama, Shuo Yang 외 arxiv

We present MMCOMET, the first multimodal commonsense knowledge graph (MMKG) that integrates physical, social, and eventive knowledge. MMCOMET extends the ATOMIC2020 knowledge graph to include a visual dimension, through …

Visual StorytellingImage CaptioningImage Retrieval

Visual Commonsense in Pretrained Unimodal and Multimodal Models

2022-05-04 · NAACL 2022 7 · Chenyu Zhang, Benjamin Van Durme, Zhuowan Li, Elias Stengel-Eskin

Our commonsense knowledge about objects includes their typical visual attributes; we know that bananas are typically yellow or green, and not purple. Text and image corpora, being subject to reporting bias, represent thi…

AttributeVisual Commonsense TestsWorld Knowledge

Reading Books is Great, But Not if You Are Driving! Visually Grounded Reasoning about Defeasible Commonsense Norms

2023-10-16 · Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang 외

Commonsense norms are defeasible by context: reading books is usually great, but not when driving a car. While contexts can be explicitly described in language, in embodied scenarios, contexts are often provided visually…

2k

MagiC: Evaluating Multimodal Cognition Toward Grounded Visual Reasoning

2025-07-09 · Chengfei Wu, Ronald Seoh, Bingxuan Li, Liqiang Zhang 외

Recent advances in large vision-language models have led to impressive performance in visual question answering and multimodal reasoning. However, it remains unclear whether these models genuinely perform grounded visual…

DiagnosticMultimodal ReasoningQuestion AnsweringVisual Question Answering+1