Textually Enriched Neural Module Networks for Visual Question Answering
Problems at the intersection of language and vision, like visual question answering, have recently been gaining a lot of attention in the field of multi-modal machine learning as computer vision research moves beyond traditional recognition tasks. There has been recent success in visual question answering using deep neural network models which use the linguistic structure of the questions to dynamically instantiate network layouts. In the process of converting the question to a network layout, the question is simplified, which results in loss of information in the model. In this paper, we enrich the image information with textual data using image captions and external knowledge bases to generate more coherent answers. We achieve 57.1% overall accuracy on the test-dev open-ended questions from the visual question answering (VQA 1.0) real image dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
DocVXQA: Context-Aware Visual Explanations for Document Question Answering
We propose DocVXQA, a novel framework for visually self-explainable document question answering. The framework is designed not only to produce accurate answers to questions but also to learn visual heatmaps that highligh…
Question AnsweringAn Empirical Evaluation of Visual Question Answering for Novel Objects
We study the problem of answering questions about images in the harder setting, where the test questions and corresponding images contain novel objects, which were not queried about in the training data. Such setting is …
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMs
Research question answering requires accurate retrieval and contextual understanding of scientific literature. However, current Retrieval-Augmented Generation (RAG) methods often struggle to balance complex document rela…
Information RetrievalQuestion AnsweringRAGRetrieval+1Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts
Visual question answering (VQA) is the task of answering questions about an image. The task assumes an understanding of both the image and the question to provide a natural language answer. VQA has gained popularity in r…
Image CaptioningLanguage ModelingLanguage ModellingQuestion Answering+3Precision Empowers, Excess Distracts: Visual Question Answering With Dynamically Infused Knowledge In Language Models
In the realm of multimodal tasks, Visual Question Answering (VQA) plays a crucial role by addressing natural language questions grounded in visual content. Knowledge-Based Visual Question Answering (KBVQA) advances this …
DecoderKnowledge GraphsLanguage ModellingQuestion Answering+2