Integrating Pretrained Language Model for Dialogue Policy Learning
Reinforcement Learning (RL) has been witnessed its potential for training a dialogue policy agent towards maximizing the accumulated rewards given from users. However, the reward can be very sparse for it is usually only provided at the end of a dialog session, which causes unaffordable interaction requirements for an acceptable dialog agent. Distinguished from many efforts dedicated to optimizing the policy and recovering the reward alternatively which suffers from easily getting stuck in local optima and model collapse, we decompose the adversarial training into two steps: 1) we integrate a pre-trained language model as a discriminator to judge whether the current system action is good enough for the last user action (i.e., \textit{next action prediction}); 2) the discriminator gives and extra local dense reward to guide the agent's exploration. The experimental result demonstrates that our method significantly improves the complete rate (~4.4\%) and success rate (~8.0\%) of the dialogue system.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingmodelReinforcement Learning (RL)Similar Papers 제목 키워드 기반
A General Highly Accurate Online Planning Method Integrating Large Language Models into Nested Rollout Policy Adaptation for Dialogue Tasks
In goal-oriented dialogue tasks, the main challenge is to steer the interaction towards a given goal within a limited number of turns. Existing approaches either rely on elaborate prompt engineering, whose effectiveness …
Prompt EngineeringDomain-Aware Dialogue State Tracker for Multi-Domain Dialogue Systems
In task-oriented dialogue systems the dialogue state tracker (DST) component is responsible for predicting the state of the dialogue based on the dialogue history. Current DST approaches rely on a predefined domain ontol…
Language ModelingLanguage ModellingTask-Oriented Dialogue SystemsDeep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning
Training a task-completion dialogue agent via reinforcement learning (RL) is costly because it requires many interactions with real users. One common alternative is to use a user simulator. However, a user simulator usua…
Reinforcement LearningReinforcement Learning (RL)Task-Completion Dialogue Policy LearningSub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning
Human conversation is inherently complex, often spanning many different topics/domains. This makes policy learning for dialogue systems very challenging. Standard flat reinforcement learning methods do not provide an eff…
Dialogue ManagementHierarchical Reinforcement LearningManagementreinforcement-learning+2CS-BERT: a pretrained model for customer service dialogues
Large-scale pretrained transformer models have demonstrated state-of-the-art (SOTA) performance in a variety of NLP tasks. Nowadays, numerous pretrained models are available in different model flavors and different langu…