Causal analysis of task completion errors in spoken music retrieval interactions
In this paper, we analyze the causes of task completion errors in spoken dialog systems, using a decision tree with N-gram features of the dialog to detect task-incomplete dialogs. The dialog for a music retrieval task is described by a sequence of tags related to user and system utterances and behaviors. The dialogs are manually classified into two classes: completed and uncompleted music retrieval tasks. Differences in tag classification performance between the two classes are discussed. We then construct decision trees which can detect if a dialog finished with the task completed or not, using information gain criterion. Decision trees using N-grams of manual tags and automatic tags achieved 74.2{\%} and 80.4{\%} classification accuracy, respectively, while the tree using interaction parameters achieved an accuracy rate of 65.7{\%}. We also discuss more details of the causality of task incompletion for spoken dialog systems using such trees.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationRetrievalTAGSimilar Papers 제목 키워드 기반
Multimodal Speech Recognition for Language-Guided Embodied Agents
Benchmarks for language-guided embodied agents typically assume text-based instructions, but deployed agents will encounter spoken instructions. While Automatic Speech Recognition (ASR) models can bridge the input gap, e…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionTraversing Knowledge Graphs in Vector Space
Path queries on a knowledge graph can be used to answer compositional questions such as "What languages are spoken by people living in Lisbon?". However, knowledge graphs often have missing facts (edges) which disrupts p…
Knowledge Base CompletionKnowledge GraphsTELEVAL: A Benchmark Designed for Spoken Language Models in Chinese Interactive Scenarios
Spoken Language Models (SLMs) are expected to support natural spoken interaction beyond task completion. However, existing SLM benchmarks primarily evaluate semantic correctness in structured settings and provide limited…
Leveraging Semantic Web Search and Browse Sessions for Multi-Turn Spoken Dialog Systems
Training statistical dialog models in spoken dialog systems (SDS) requires large amounts of annotated data. The lack of scalable methods for data mining and annotation poses a significant hurdle for state-of-the-art stat…
Entity Extraction using GANRelation ExtractionAre cascade dialogue state tracking models speaking out of turn in spoken dialogues?
In Task-Oriented Dialogue (TOD) systems, correctly updating the system's understanding of the user's needs is key to a smooth interaction. Traditionally TOD systems are composed of several modules that interact with one …
Dialogue State Tracking