Fully Authentic Visual Question Answering Dataset from Online Communities
Visual Question Answering (VQA) entails answering questions about images. We introduce the first VQA dataset in which all contents originate from an authentic use case. Sourced from online question answering community forums, we call it VQAonline. We characterize this dataset and how it relates to eight mainstream VQA datasets. Observing that answers in our dataset tend to be much longer (i.e., a mean of 173 words) and so incompatible with standard VQA evaluation metrics, we instead utilize popular metrics for longer text evaluation for evaluating six state-of-the-art VQA models on VQAonline and report where they struggle most. Finally, we analyze which evaluation metrics align best with human judgments. To facilitate future extensions, we publicly-share the dataset at: https://vqaonline.github.io/.
Code (1)
Tasks
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Chart Question Answering from Real-World Analytical Narratives
We present a new dataset for chart question answering (CQA) constructed from visualization notebooks. The dataset features real-world, multi-view charts paired with natural language questions grounded in analytical narra…
Chart Question AnsweringAn Evaluation of GPT-4V and Gemini in Online VQA
While there is much excitement about the potential of large multimodal models (LMM), a comprehensive evaluation is critical to establish their true capabilities and limitations. In support of this aim, we evaluate two st…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)WearVQA: A Visual Question Answering Benchmark for Wearables in Egocentric Authentic Real-world scenarios
We introduce WearVQA, the first benchmark specifically designed to evaluate the Visual Question Answering (VQA) capabilities of multi-model AI assistant on wearable devices like smart glasses. Unlike prior benchmarks tha…
Visual Question AnsweringTransfer Learning via Unsupervised Task Discovery for Visual Question Answering
We study how to leverage off-the-shelf visual and linguistic data to cope with out-of-vocabulary answers in visual question answering task. Existing large-scale visual datasets with annotations such as image class labels…
Question AnsweringTransfer LearningVisual Question AnsweringVisual Question Answering (VQA)Check It Again: Progressive Visual Question Answering via Visual Entailment
While sophisticated Visual Question Answering models have achieved remarkable success, they tend to answer questions only according to superficial correlations between question and answer. Several recent approaches have …
Question AnsweringVisual EntailmentVisual Question AnsweringVisual Question Answering (VQA)