Benchmark Data and Evaluation Framework for Intent Discovery Around COVID-19 Vaccine Hesitancy
The COVID-19 pandemic has made a huge global impact and cost millions of lives. As COVID-19 vaccines were rolled out, they were quickly met with widespread hesitancy. To address the concerns of hesitant people, we launched VIRA, a public dialogue system aimed at addressing questions and concerns surrounding the COVID-19 vaccines. Here, we release VIRADialogs, a dataset of over 8k dialogues conducted by actual users with VIRA, providing a unique real-world conversational dataset. In light of rapid changes in users' intents, due to updates in guidelines or in response to new information, we highlight the important task of intent discovery in this use-case. We introduce a novel automatic evaluation framework for intent discovery, leveraging the existing intent classifier of VIRA. We use this framework to report baseline intent discovery results over VIRADialogs, that highlight the difficulty of this task.
Code (1)
Tasks
8kIntent DiscoverySimilar Papers 제목 키워드 기반
TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition
TEXTOIR is the first integrated and visualized platform for text open intent recognition. It is composed of two main modules: open intent detection and open intent discovery. Each module integrates most of the state-of-t…
Intent DetectionIntent DiscoveryIntent RecognitionManagement+2A Diffusion Weighted Graph Framework for New Intent Discovery
New Intent Discovery (NID) aims to recognize both new and known intents from unlabeled data with the aid of limited labeled data containing only known intents. Without considering structure relationships between samples,…
Contrastive LearningIntent DiscoveryA Clustering Framework for Unsupervised and Semi-supervised New Intent Discovery
New intent discovery is of great value to natural language processing, allowing for a better understanding of user needs and providing friendly services. However, most existing methods struggle to capture the complicated…
ClusteringIntent DiscoveryRepresentation LearningSemantic Similarity+1Towards Intelligent Geospatial Data Discovery: a knowledge graph-driven multi-agent framework powered by large language models
The rapid growth in the volume, variety, and velocity of geospatial data has created data ecosystems that are highly distributed, heterogeneous, and semantically inconsistent. Existing data catalogs, portals, and infrast…
Large Language Models Meet Open-World Intent Discovery and Recognition: An Evaluation of ChatGPT
The tasks of out-of-domain (OOD) intent discovery and generalized intent discovery (GID) aim to extend a closed intent classifier to open-world intent sets, which is crucial to task-oriented dialogue (TOD) systems. Previ…
In-Context LearningIntent Discovery