Learning to Disambiguate by Asking Discriminative Questions
The ability to ask questions is a powerful tool to gather information in order to learn about the world and resolve ambiguities. In this paper, we explore a novel problem of generating discriminative questions to help disambiguate visual instances. Our work can be seen as a complement and new extension to the rich research studies on image captioning and question answering. We introduce the first large-scale dataset with over 10,000 carefully annotated images-question tuples to facilitate benchmarking. In particular, each tuple consists of a pair of images and 4.6 discriminative questions (as positive samples) and 5.9 non-discriminative questions (as negative samples) on average. In addition, we present an effective method for visual discriminative question generation. The method can be trained in a weakly supervised manner without discriminative images-question tuples but just existing visual question answering datasets. Promising results are shown against representative baselines through quantitative evaluations and user studies.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingImage CaptioningQuestion AnsweringQuestion GenerationQuestion-GenerationVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Incremental Object Grounding Using Scene Graphs
Object grounding tasks aim to locate the target object in an image through verbal communications. Understanding human command is an important process needed for effective human-robot communication. However, this is chall…
ObjectResolving Intent Ambiguities by Retrieving Discriminative Clarifying Questions
Task oriented Dialogue Systems generally employ intent detection systems in order to map user queries to a set of pre-defined intents. However, user queries appearing in natural language can be easily ambiguous and hence…
Intent DetectionQuestion GenerationQuestion-GenerationTask-Oriented Dialogue SystemsAsking Clarification Questions to Handle Ambiguity in Open-Domain QA
Ambiguous questions persist in open-domain question answering, because formulating a precise question with a unique answer is often challenging. Previously, Min et al. (2020) have tackled this issue by generating disambi…
Open-Domain Question AnsweringQuestion AnsweringKnowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
User queries are often underspecified and may admit multiple valid interpretations. Rather than silently making assumptions about the user's intent, a helpful assistant should surface such ambiguity by asking a clarifyin…
Question AnsweringIRIS: Intent Resolution via Inference-time Saccades for Open-Ended VQA in Large Vision-Language Models
We introduce IRIS (Intent Resolution via Inference-time Saccades), a novel training-free approach that uses eye-tracking data in real-time to resolve ambiguity in open-ended VQA. Through a comprehensive user study with 5…