QAConv: Question Answering on Informative Conversations
This paper introduces QAConv, a new question answering (QA) dataset that uses conversations as a knowledge source. We focus on informative conversations, including business emails, panel discussions, and work channels. Unlike open-domain and task-oriented dialogues, these conversations are usually long, complex, asynchronous, and involve strong domain knowledge. In total, we collect 34,608 QA pairs from 10,259 selected conversations with both human-written and machine-generated questions. We use a question generator and a dialogue summarizer as auxiliary tools to collect and recommend questions. The dataset has two testing scenarios: chunk mode and full mode, depending on whether the grounded partial conversation is provided or retrieved. Experimental results show that state-of-the-art pretrained QA systems have limited zero-shot performance and tend to predict our questions as unanswerable. Our dataset provides a new training and evaluation testbed to facilitate QA on conversations research.
Code (1)
Tasks
Question AnsweringSimilar Papers 제목 키워드 기반
QAConv: Question Answering on Informative Conversations
This paper introduces QAConv, a new question answering (QA) dataset that uses conversations as a knowledge source. We focus on informative conversations, including business emails, panel discussions, and work channels. U…
Question AnsweringStay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations
We investigate the problem of generating informative questions in information-asymmetric conversations. Unlike previous work on question generation which largely assumes knowledge of what the answer might be, we are inte…
InformativenessQuestion GenerationQuestion-GenerationReinforcement Learning+1HealthQ: Unveiling Questioning Capabilities of LLM Chains in Healthcare Conversations
In digital healthcare, large language models (LLMs) have primarily been utilized to enhance question-answering capabilities and improve patient interactions. However, effective patient care necessitates LLM chains that c…
Dataset GenerationInformativenessnamed-entity-recognitionNamed Entity Recognition+5Unsupervised Domain Adaptation on Question-Answering System with Conversation Data
Machine reading comprehension (MRC) is a task for question answering that finds answers to questions from documents of knowledge. Most studies on the domain adaptation of MRC require documents describing knowledge of the…
Domain AdaptationMachine Reading ComprehensionQuestion AnsweringReading Comprehension+1Ditch the Gold Standard: Re-evaluating Conversational Question Answering
Conversational question answering aims to provide natural-language answers to users in information-seeking conversations. Existing conversational QA benchmarks compare models with pre-collected human-human conversations,…
Conversational Question AnsweringQuestion AnsweringQuestion Rewriting