paper-with-me

홈 › Papers

Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation

2026-05-05 · Quanxing Xu, Ling Zhou, Xian Zhong, Xiaohua Huang, Rubing Huang, Chia-Wen Lin arxiv

With advances in multimodal research and deep learning, Multimodal Large Language Models (MLLMs) have emerged as a powerful paradigm for a wide range of multimodal tasks. As a core problem in vision-language research, Visual Question Answering (VQA) has increasingly employed MLLMs to improve performance, particularly in open-domain settings where external knowledge is essential. In this work, we aim to further enhance retrieval-based VQA by more effectively integrating MLLMs with structured reasoning and knowledge acquisition. We introduce a logical prompting strategy that fuses Chain-of-Thought (CoT) reasoning with Visual Question Decomposition (VQD), termed CoVQD, to guide retrieval toward more accurate and relevant knowledge for MLLM inference. Building on this idea, we propose a new framework, CoVQD-guided RAG (CgRAG), which enables MLLMs to access more comprehensive and coherent external knowledge while benefiting from structured visual-text reasoning guidance, thereby improving generalization and reliability in complex cross-domain VQA scenarios. Extensive experiments on E-VQA, InfoSeek, and OKVQA benchmarks demonstrate the effectiveness of the proposed method.

📄 PDF Abstract BibTeX arXiv:2605.03790

Code (0)

등록된 구현이 없습니다.

Tasks

Visual Question Answering

Similar Papers 제목 키워드 기반

LOVA3: Learning to Visual Question Answering, Asking and Assessment

2024-05-23 · Henry Hengyuan Zhao, Pan Zhou, Difei Gao, Zechen Bai 외

Question answering, asking, and assessment are three innate human traits crucial for understanding the world and acquiring knowledge. By enhancing these capabilities, humans can more effectively utilize data, leading to …

Question AnsweringVisual Question Answering

CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering

2025-03-01 · CVPR 2025 1 · Tianyu Huai, Jie zhou, Xingjiao Wu, Qin Chen 외

Multimodal large language models (MLLMs) have garnered widespread attention from researchers due to their remarkable understanding and generation capabilities in visual language tasks (e.g., visual question answering). H…

Continual LearningLanguage ModelingLanguage ModellingLarge Language Model+5

Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queries

2025-02-23 · Yin Wu, Quanyu Long, Jing Li, Jianfei Yu 외

Retrieval-Augmented Generation (RAG) is a popular approach for enhancing Large Language Models (LLMs) by addressing their limitations in verifying facts and answering knowledge-intensive questions. As the research in LLM…

BenchmarkingImage RetrievalQuestion AnsweringRAG+2

MEGC2026: Micro-Expression Grand Challenge on Visual Question Answering

2026-03-09 · Xinqi Fan, Jingting Li, John See, Moi Hoon Yap 외 arxiv

Facial micro-expressions (MEs) are involuntary movements of the face that occur spontaneously when a person experiences an emotion but attempts to suppress or repress the facial expression, typically found in a high-stak…

Visual Question AnsweringVideo Question AnsweringMultimodal Reasoning

Visual Question Answering Instruction: Unlocking Multimodal Large Language Model To Domain-Specific Visual Multitasks

2024-02-13 · Jusung Lee, Sungguk Cha, Younghyun Lee, Cheoljong Yang

Having revolutionized natural language processing (NLP) applications, large language models (LLMs) are expanding into the realm of multimodal inputs. Owing to their ability to interpret images, multimodal LLMs (MLLMs) ha…

Language ModelingLanguage ModellingLarge Language ModelMultimodal Large Language Model+3