OLISIA: a Cascade System for Spoken Dialogue State Tracking
Though Dialogue State Tracking (DST) is a core component of spoken dialogue systems, recent work on this task mostly deals with chat corpora, disregarding the discrepancies between spoken and written language.In this paper, we propose OLISIA, a cascade system which integrates an Automatic Speech Recognition (ASR) model and a DST model. We introduce several adaptations in the ASR and DST modules to improve integration and robustness to spoken conversations.With these adaptations, our system ranked first in DSTC11 Track 3, a benchmark to evaluate spoken DST. We conduct an in-depth analysis of the results and find that normalizing the ASR outputs and adapting the DST inputs through data augmentation, along with increasing the pre-trained models size all play an important role in reducing the performance discrepancy between written and spoken conversations.
Code (1)
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Data AugmentationDialogue State Trackingspeech-recognitionSpeech RecognitionSpoken Dialogue SystemsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Are cascade dialogue state tracking models speaking out of turn in spoken dialogues?
In Task-Oriented Dialogue (TOD) systems, correctly updating the system's understanding of the user's needs is key to a smooth interaction. Traditionally TOD systems are composed of several modules that interact with one …
Dialogue State TrackingWavChat: A Survey of Spoken Dialogue Models
Recent advancements in spoken dialogue models, exemplified by systems like GPT-4o, have captured significant attention in the speech domain. Compared to traditional three-tier cascaded spoken dialogue models that compris…
speech-recognitionSpeech RecognitionSpoken Dialogue SystemsSurvey+2Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue
An avatar that holds a conversation should decide what to say and to move while saying it, yet these abilities live in separate model families: spoken dialogue models produce speech without motion, and co-speech motion m…
Is one brick enough to break the wall of spoken dialogue state tracking?
In Task-Oriented Dialogue (TOD) systems, correctly updating the system's understanding of the user's requests (\textit{a.k.a} dialogue state tracking) is key to a smooth interaction. Traditionally, TOD systems perform th…
Dialogue State TrackingESPnet-SDS: Unified Toolkit and Demo for Spoken Dialogue Systems
Advancements in audio foundation models (FMs) have fueled interest in end-to-end (E2E) spoken dialogue systems, but different web interfaces for each system makes it challenging to compare and contrast them effectively. …
DiversitySpoken Dialogue Systems