In-Context Learning User Simulators for Task-Oriented Dialog Systems
This paper presents a novel application of large language models in user simulation for task-oriented dialog systems, specifically focusing on an in-context learning approach. By harnessing the power of these models, the proposed approach generates diverse utterances based on user goals and limited dialog examples. Unlike traditional simulators, this method eliminates the need for labor-intensive rule definition or extensive annotated data, making it more efficient and accessible. Additionally, an error analysis of the interaction between the user simulator and dialog system uncovers common mistakes, providing valuable insights into areas that require improvement. Our implementation is available at https://github.com/telepathylabsai/prompt-based-user-simulator.
Code (2)
Tasks
Goal-Oriented Dialogue SystemsIn-Context LearningInteractive Evaluation of DialogTask-Oriented Dialogue SystemsUser SimulationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systems
Task-oriented dialogue systems (TDSs) are assessed mainly in an offline setting or through human evaluation. The evaluation is often limited to single-turn or is very time-intensive. As an alternative, user simulators th…
Conversational RecommendationTask-Oriented Dialogue SystemsDuetSim: Building User Simulator with Dual Large Language Models for Task-Oriented Dialogues
User Simulators play a pivotal role in training and evaluating task-oriented dialogue systems. Traditional user simulators typically rely on human-engineered agendas, resulting in generated responses that often lack dive…
DiversityResponse GenerationTask-Oriented Dialogue SystemsMUST: A Framework for Training Task-oriented Dialogue Systems with Multiple User SimulaTors
Recent works try to optimize a Task-oriented Dialogue System with reinforcement learning (RL) by building user simulators. However, most of them only focus on training the dialogue system using a single user simulator. I…
Reinforcement Learning (RL)Task-Oriented Dialogue SystemsGenerating Diverse Personas for User Simulators to Test Interview Dialogue Systems
This paper addresses the issue of the significant labor required to test interview dialogue systems. While interview dialogue systems are expected to be useful in various scenarios, like other dialogue systems, testing t…
Task-Oriented Dialogue SystemsNeural User Simulation for Corpus-based Policy Optimisation of Spoken Dialogue Systems
User Simulators are one of the major tools that enable offline training of task-oriented dialogue systems. For this task the Agenda-Based User Simulator (ABUS) is often used. The ABUS is based on hand-crafted rules and i…
Dialogue ManagementDiversityReinforcement LearningSpoken Dialogue Systems+2