Wizard of Tasks: A Novel Conversational Dataset for Solving Real-World Tasks in Conversational Settings
Conversational Task Assistants (CTAs) are conversational agents whose goal is to help humans perform real-world tasks. CTAs can help in exploring available tasks, answering task-specific questions and guiding users through step-by-step instructions. In this work, we present Wizard of Tasks, the first corpus of such conversations in two domains: Cooking and Home Improvement. We crowd-sourced a total of 549 conversations (18,077 utterances) with an asynchronous Wizard-of-Oz setup, relying on recipes from WholeFoods Market for the cooking domain, and WikiHow articles for the home improvement domain. We present a detailed data analysis and show that the collected data can be a valuable and challenging resource for CTAs in two tasks: Intent Classification (IC) and Abstractive Question Answering (AQA). While on IC we acquired a high performing model (>85% F1), on AQA the performance is far from being satisfactory (~27% BertScore-F1), suggesting that more work is needed to solve the task of low-resource AQA.
Code (0)
등록된 구현이 없습니다.
Tasks
abstractive question answeringArticlesintent-classificationIntent ClassificationQuestion AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RecWizard: A Toolkit for Conversational Recommendation with Modular, Portable Models and Interactive User Interface
We present a new Python toolkit called RecWizard for Conversational Recommender Systems (CRS). RecWizard offers support for development of models and interactive user interface, drawing from the best practices of the Hug…
Conversational RecommendationRecommendation SystemsWhy and When: Understanding System Initiative during Conversational Collaborative Search
In the last decade, conversational search has attracted considerable attention. However, most research has focused on systems that can support a \emph{single} searcher. In this paper, we explore how systems can support \…
Conversational SearchTWIZ-v2: The Wizard of Multimodal Conversational-Stimulus
In this report, we describe the vision, challenges, and scientific contributions of the Task Wizard team, TWIZ, in the Alexa Prize TaskBot Challenge 2022. Our vision, is to build TWIZ bot as an helpful, multimodal, knowl…
Language ModelingLanguage ModellingLarge Language ModelUsers and Wizards in Conversations: How WoZ Interface Choices Define Human-Robot Interactions
In this paper, we investigated how the choice of a Wizard-of-Oz (WoZ) interface affects communication with a robot from both the user's and the wizard's perspective. In a conversational setting, we used three WoZ interfa…
CRWIZ: A Framework for Crowdsourcing Real-Time Wizard-of-Oz Dialogues
Large corpora of task-based and open-domain conversational dialogues are hugely valuable in the field of data-driven dialogue systems. Crowdsourcing platforms, such as Amazon Mechanical Turk, have been an effective metho…