Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems
In the realm of dialogue systems, user simulation techniques have emerged as a game-changer, redefining the evaluation and enhancement of task-oriented dialogue (TOD) systems. These methods are crucial for replicating real user interactions, enabling applications like synthetic data augmentation, error detection, and robust evaluation. However, existing approaches often rely on rigid rule-based methods or on annotated data. This paper introduces DAUS, a Domain-Aware User Simulator. Leveraging large language models, we fine-tune DAUS on real examples of task-oriented dialogues. Results on two relevant benchmarks showcase significant improvements in terms of user goal fulfillment. Notably, we have observed that fine-tuning enhances the simulator's coherence with user goals, effectively mitigating hallucinations -- a major source of inconsistencies in simulator responses.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationTask-Oriented Dialogue SystemsUser SimulationSimilar Papers 제목 키워드 기반
A Corpus-free State2Seq User Simulator for Task-oriented Dialogue
Recent reinforcement learning algorithms for task-oriented dialogue system absorbs a lot of interest. However, an unavoidable obstacle for training such algorithms is that annotated dialogue corpora are often unavailable…
DiversityReinforcement LearningMUST: A Framework for Training Task-oriented Dialogue Systems with Multiple User SimulaTors
Recent works try to optimize a Task-oriented Dialogue System with reinforcement learning (RL) by building user simulators. However, most of them only focus on training the dialogue system using a single user simulator. I…
Reinforcement Learning (RL)Task-Oriented Dialogue SystemsMetaphorical User Simulators for Evaluating Task-oriented Dialogue Systems
Task-oriented dialogue systems (TDSs) are assessed mainly in an offline setting or through human evaluation. The evaluation is often limited to single-turn or is very time-intensive. As an alternative, user simulators th…
Conversational RecommendationTask-Oriented Dialogue SystemsSpokenUS: A Spoken User Simulator for Task-Oriented Dialogue
Robust task-oriented spoken dialogue agents require exposure to the full diversity of how people interact through speech. Building spoken user simulators that address this requires large-scale spoken task-oriented dialog…
User Modeling for Task Oriented Dialogues
We introduce end-to-end neural network based models for simulating users of task-oriented dialogue systems. User simulation in dialogue systems is crucial from two different perspectives: (i) automatic evaluation of diff…
Dialogue State TrackingDiversityTask-Oriented Dialogue SystemsUser Simulation