Model-based Bayesian Reinforcement Learning for Dialogue Management

2013-04-05 · Pierre Lison

Reinforcement learning methods are increasingly used to optimise dialogue policies from experience. Most current techniques are model-free: they directly estimate the utility of various actions, without explicit model of the interaction dynamics. In this paper, we investigate an alternative strategy grounded in model-based Bayesian reinforcement learning. Bayesian inference is used to maintain a posterior distribution over the model parameters, reflecting the model uncertainty. This parameter distribution is gradually refined as more data is collected and simultaneously used to plan the agent's actions. Within this learning framework, we carried out experiments with two alternative formalisations of the transition model, one encoded with standard multinomial distributions, and one structured with probabilistic rules. We demonstrate the potential of our approach with empirical results on a user simulator constructed from Wizard-of-Oz data in a human-robot interaction scenario. The results illustrate in particular the benefits of capturing prior domain knowledge with high-level rules.

📄 PDF Abstract BibTeX arXiv:1304.1819

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian InferenceDialogue ManagementManagementmodelreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Uncertainty Estimates for Efficient Neural Network-based Dialogue Policy Optimisation

2017-11-30 · Christopher Tegho, Paweł Budzianowski, Milica Gašić

In statistical dialogue management, the dialogue manager learns a policy that maps a belief state to an action for the system to perform. Efficient exploration is key to successful policy optimisation. Current deep reinf…

Deep Reinforcement Learning Dialogue Management Efficient Exploration Efficient Neural Network+2

Distributed Structured Actor-Critic Reinforcement Learning for Universal Dialogue Management

2020-09-22 · Zhi Chen, Lu Chen, Xiaoyuan Liu, Kai Yu

The task-oriented spoken dialogue system (SDS) aims to assist a human user in accomplishing a specific task (e.g., hotel booking). The dialogue management is a core part of SDS. There are two main missions in dialogue ma…

Decision Making Deep Reinforcement Learning Dialogue Management Management+3

Curriculum Learning Based on Reward Sparseness for Deep Reinforcement Learning of Task Completion Dialogue Management

2018-10-01 · WS 2018 10 · Atsushi Saito

Learning from sparse and delayed reward is a central issue in reinforcement learning. In this paper, to tackle reward sparseness problem of task oriented dialogue management, we propose a curriculum based approach on the…

Deep Reinforcement Learning Dialogue Management Information Retrieval Management+5

Continuously Learning Neural Dialogue Management

2016-06-08 · Pei-Hao Su, Milica Gasic, Nikola Mrksic, Lina Rojas-Barahona 외

We describe a two-step approach for dialogue management in task-oriented spoken dialogue systems. A unified neural network framework is proposed to enable the system to first learn by supervision from a set of dialogue d…

Dialogue Management Management reinforcement-learning Reinforcement Learning+3

Sub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning

2017-06-19 · WS 2017 8 · Paweł Budzianowski, Stefan Ultes, Pei-Hao Su, Nikola Mrkšić 외

Human conversation is inherently complex, often spanning many different topics/domains. This makes policy learning for dialogue systems very challenging. Standard flat reinforcement learning methods do not provide an eff…

Dialogue Management Hierarchical Reinforcement Learning Management reinforcement-learning+2