Question Relevance in VQA: Identifying Non-Visual And False-Premise Questions
Visual Question Answering (VQA) is the task of answering natural-language questions about images. We introduce the novel problem of determining the relevance of questions to images in VQA. Current VQA models do not reason about whether a question is even related to the given image (e.g. What is the capital of Argentina?) or if it requires information from external resources to answer correctly. This can break the continuity of a dialogue in human-machine interaction. Our approaches for determining relevance are composed of two stages. Given an image and a question, (1) we first determine whether the question is visual or not, (2) if visual, we determine whether the question is relevant to the given image or not. Our approaches, based on LSTM-RNNs, VQA model uncertainty, and caption-question similarity, are able to outperform strong baselines on both relevance tasks. We also present human studies showing that VQA models augmented with such question relevance reasoning are perceived as more intelligent, reasonable, and human-like.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringQuestion SimilarityVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
The Promise of Premise: Harnessing Question Premises in Visual Question Answering
In this paper, we make a simple observation that questions about images often contain premises - objects and relationships implied by the question - and that reasoning about premises can help Visual Question Answering (V…
Question AnsweringRelevance DetectionVisual Question AnsweringVisual Question Answering (VQA)Won't Get Fooled Again: Answering Questions with False Premises
Pre-trained language models (PLMs) have shown unprecedented potential in various fields, especially as the backbones for question-answering (QA) systems. However, they tend to be easily deceived by tricky questions such …
Question AnsweringJudge Before Answer: Can MLLM Discern the False Premise in Question?
Multimodal large language models (MLLMs) have witnessed astonishing advancements in recent years. Despite these successes, MLLMs remain vulnerable to flase premise problems. However, existing benchmarks targeting this is…
Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Models
Large Language Models (LLMs) have shown impressive capabilities but still suffer from the issue of hallucinations. A significant type of this issue is the false premise hallucination, which we define as the phenomenon wh…
HallucinationMultiHoax: A Dataset of Multi-hop False-Premise Questions
As Large Language Models are increasingly deployed in high-stakes domains, their ability to detect false assumptions and reason critically is crucial for ensuring reliable outputs. False-premise questions (FPQs) serve as…