Latent Variable Models for Visual Question Answering
Current work on Visual Question Answering (VQA) explore deterministic approaches conditioned on various types of image and question features. We posit that, in addition to image and question pairs, other modalities are useful for teaching machine to carry out question answering. Hence in this paper, we propose latent variable models for VQA where extra information (e.g. captions and answer categories) are incorporated as latent variables, which are observed during training but in turn benefit question-answering performance at test time. Experiments on the VQA v2.0 benchmarking dataset demonstrate the effectiveness of our proposed models: they improve over strong baselines, especially those that do not rely on extensive language-vision pre-training.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Multi-modality Latent Interaction Network for Visual Question Answering
Exploiting relationships between visual regions and question words have achieved great success in learning multi-modality features for Visual Question Answering (VQA). However, we argue that existing methods mostly model…
Language ModelingLanguage ModellingQuestion AnsweringVisual Question Answering+1Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering
We propose a new class of probabilistic neural-symbolic models, that have symbolic functional programs as a latent, stochastic variable. Instantiated in the context of visual question answering, our probabilistic formula…
counterfactualQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering
We train a language model (LM) to robustly answer multistep questions by generating and answering sub-questions. We propose Chain-of-Questions, a framework that trains a model to generate sub-questions and sub-answers on…
Language ModelingLanguage ModellingQuestion AnsweringThink Visually: Question Answering through Virtual Imagery
In this paper, we study the problem of geometric reasoning in the context of question-answering. We introduce Dynamic Spatial Memory Network (DSMN), a new deep network architecture designed for answering questions that a…
Question AnsweringVisual Commonsense ReasoningImproving Span-based Question Answering Systems with Coarsely Labeled Data
We study approaches to improve fine-grained short answer Question Answering models by integrating coarse-grained data annotated for paragraph-level relevance and show that coarsely annotated data can bring significant pe…
Multi-Task LearningQuestion Answering