Why Guided Dialog Policy Learning performs well? Understanding the role of adversarial learning and its alternative
Dialog policies, which determine a system's action based on the current state at each dialog turn, are crucial to the success of the dialog. In recent years, reinforcement learning (RL) has emerged as a promising option for dialog policy learning (DPL). In RL-based DPL, dialog policies are updated according to rewards. The manual construction of fine-grained rewards, such as state-action-based ones, to effectively guide the dialog policy is challenging in multi-domain task-oriented dialog scenarios with numerous state-action pair combinations. One way to estimate rewards from collected data is to train the reward estimator and dialog policy simultaneously using adversarial learning (AL). Although this method has demonstrated superior performance experimentally, it is fraught with the inherent problems of AL, such as mode collapse. This paper first identifies the role of AL in DPL through detailed analyses of the objective functions of dialog policy and reward estimator. Next, based on these analyses, we propose a method that eliminates AL from reward estimation and DPL while retaining its advantages. We evaluate our method using MultiWOZ, a multi-domain task-oriented dialog corpus.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement Learning (RL)Similar Papers 제목 키워드 기반
Using Reinforcement Learning to Model Incrementality in a Fast-Paced Dialogue Game
We apply Reinforcement Learning (RL) to the problem of incremental dialogue policy learning in the context of a fast-paced dialogue game. We compare the policy learned by RL with a high-performance baseline policy which …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Spoken Dialogue SystemsTowards Policy-Guided Conversational Recommendation with Dialogue Acts
Conversation Recommender System (CRS) aims to recommend items through nature conversation. Existing works in open-ended CRS mainly focus on recommendation and generation, but lacks of control over dialogue policy. In add…
Conversational RecommendationRecommendation SystemsResponse GenerationSpeaker Role Contextual Modeling for Language Understanding and Dialogue Policy Learning
Language understanding (LU) and dialogue policy learning are two essential components in conversational systems. Human-human dialogues are not well-controlled and often random and unpredictable due to their own goals and…
Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog
Dialog policy decides what and how a task-oriented dialog system will respond, and plays a vital role in delivering effective conversations. Many studies apply Reinforcement Learning to learn a dialog policy with the rew…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Policy-driven Knowledge Selection and Response Generation for Document-grounded Dialogue
Document-grounded dialogue (DGD) uses documents as external knowledge for dialogue generation. Correctly understanding the dialogue context is crucial for selecting knowledge from the document and generating proper respo…
Dialogue GenerationDialogue UnderstandingResponse Generation