A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding
Zero-shot dialogue understanding aims to enable dialogue to track the user's needs without any training data, which has gained increasing attention. In this work, we investigate the understanding ability of ChatGPT for zero-shot dialogue understanding tasks including spoken language understanding (SLU) and dialogue state tracking (DST). Experimental results on four popular benchmarks reveal the great potential of ChatGPT for zero-shot dialogue understanding. In addition, extensive analysis shows that ChatGPT benefits from the multi-turn interactive prompt in the DST task but struggles to perform slot filling for SLU. Finally, we summarize several unexpected behaviors of ChatGPT in dialogue understanding tasks, hoping to provide some insights for future research on building zero-shot dialogue understanding systems with Large Language Models (LLMs).
Code (0)
등록된 구현이 없습니다.
Tasks
Dialogue State TrackingDialogue Understandingslot-fillingSlot FillingSpoken Language UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?
Recent research on dialogue state tracking (DST) focuses on methods that allow few- and zero-shot transfer to new domains or schemas. However, performance gains heavily depend on aggressive data augmentation and fine-tun…
Data AugmentationDialogue State TrackingIn-Context LearningLanguage Modeling+1Empirical Evaluation of ChatGPT on Requirements Information Retrieval Under Zero-Shot Setting
Recently, various illustrative examples have shown the impressive ability of generative large language models (LLMs) to perform NLP related tasks. ChatGPT undoubtedly is the most representative model. We empirically eval…
Information RetrievalLanguage ModellingRetrievalTowards LLM-driven Dialogue State Tracking
Dialogue State Tracking (DST) is of paramount importance in ensuring accurate tracking of user goals and system actions within task-oriented dialogue systems. The emergence of large language models (LLMs) such as GPT3 an…
Dialogue State TrackingTask-Oriented Dialogue SystemsChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations
This paper aims to quantitatively evaluate the performance of ChatGPT, an interactive large language model, on inter-sentential relations such as temporal relations, causal relations, and discourse relations. Given ChatG…
Discourse ParsingIn-Context LearningLanguage ModelingLanguage Modelling+5Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences
As a natural language assistant, ChatGPT is capable of performing various tasks, including but not limited to article generation, code completion, and data analysis. Furthermore, ChatGPT has consistently demonstrated a r…
Code Completion