Towards LLM-driven Dialogue State Tracking
Dialogue State Tracking (DST) is of paramount importance in ensuring accurate tracking of user goals and system actions within task-oriented dialogue systems. The emergence of large language models (LLMs) such as GPT3 and ChatGPT has sparked considerable interest in assessing their efficacy across diverse applications. In this study, we conduct an initial examination of ChatGPT's capabilities in DST. Our evaluation uncovers the exceptional performance of ChatGPT in this task, offering valuable insights to researchers regarding its capabilities and providing useful directions for designing and enhancing dialogue systems. Despite its impressive performance, ChatGPT has significant limitations including its closed-source nature, request restrictions, raising data privacy concerns, and lacking local deployment capabilities. To address these concerns, we present LDST, an LLM-driven DST framework based on smaller, open-source foundation models. By utilizing a novel domain-slot instruction tuning method, LDST achieves performance on par with ChatGPT. Comprehensive evaluations across three distinct experimental settings, we find that LDST exhibits remarkable performance improvements in both zero-shot and few-shot setting compared to previous SOTA methods. The source code is provided for reproducibility.
Code (1)
Tasks
Dialogue State TrackingTask-Oriented Dialogue SystemsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Intent-driven In-context Learning for Few-shot Dialogue State Tracking
Dialogue state tracking (DST) plays an essential role in task-oriented dialogue systems. However, user's input may contain implicit information, posing significant challenges for DST tasks. Additionally, DST data include…
Dialogue State TrackingIn-Context LearningLanguage ModelingLanguage Modelling+2Multi-Domain Dialogue State Tracking with Top-K Slot Self Attention
As an important component of task-oriented dialogue systems, dialogue state tracking is designed to track the dialogue state through the conversations between users and systems. Multi-domain dialogue state tracking is a …
Dialogue State TrackingMulti-domain Dialogue State TrackingTask-Oriented Dialogue SystemsA Robust Data-Driven Approach for Dialogue State Tracking of Unseen Slot Values
A Dialogue State Tracker is a key component in dialogue systems which estimates the beliefs of possible user goals at each dialogue turn. Deep learning approaches using recurrent neural networks have shown state-of-the-a…
Dialogue State TrackingFully Statistical Neural Belief Tracking
This paper proposes an improvement to the existing data-driven Neural Belief Tracking (NBT) framework for Dialogue State Tracking (DST). The existing NBT model uses a hand-crafted belief state update mechanism which invo…
Dialogue State TrackingFully Statistical Neural Belief Tracking
This paper proposes an improvement to the existing data-driven Neural Belief Tracking (NBT) framework for Dialogue State Tracking (DST). The existing NBT model uses a hand-crafted belief state update mechanism which invo…
Dialogue ManagementDialogue State TrackingSpoken Language UnderstandingWord Embeddings