Asking Clarification Questions in Knowledge-Based Question Answering
The ability to ask clarification questions is essential for knowledge-based question answering (KBQA) systems, especially for handling ambiguous phenomena. Despite its importance, clarification has not been well explored in current KBQA systems. Further progress requires supervised resources for training and evaluation, and powerful models for clarification-related text understanding and generation. In this paper, we construct a new clarification dataset, CLAQUA, with nearly 40K open-domain examples. The dataset supports three serial tasks: given a question, identify whether clarification is needed; if yes, generate a clarification question; then predict answers base on external user feedback. We provide representative baselines for these tasks and further introduce a coarse-to-fine model for clarification question generation. Experiments show that the proposed model achieves better performance than strong baselines. The further analysis demonstrates that our dataset brings new challenges and there still remain several unsolved problems, like reasonable automatic evaluation metrics for clarification question generation and powerful models for handling entity sparsity.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringQuestion GenerationQuestion-GenerationSimilar Papers 제목 키워드 기반
Asking Clarification Questions to Handle Ambiguity in Open-Domain QA
Ambiguous questions persist in open-domain question answering, because formulating a precise question with a unique answer is often challenging. Previously, Min et al. (2020) have tackled this issue by generating disambi…
Open-Domain Question AnsweringQuestion AnsweringTeaching Vision-Language Models to Ask: Resolving Ambiguity in Visual Questions
In visual question answering (VQA) context, users often pose ambiguous questions to visual language models (VLMs) due to varying expression habits. Existing research addresses such ambiguities primarily by rephrasing que…
Visual Question AnsweringCuriosity by Design: An LLM-based Coding Assistant Asking Clarification Questions
Large Language Models (LLMs) are increasingly used as coding assistants. However, the ambiguity of the developer's prompt often leads to incorrect code generation, as current models struggle to infer user intent without …
Prompt EngineeringCode GenerationAsking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions
Clarification questions are an essential dialogue tool to signal misunderstanding, ambiguities, and under-specification in language use. While humans are able to resolve uncertainty by asking questions since childhood, m…
Open-domain clarification question generation without question examples
An overarching goal of natural language processing is to enable machines to communicate seamlessly with humans. However, natural language can be ambiguous or unclear. In cases of uncertainty, humans engage in an interact…
Question GenerationQuestion-Generation