Sample-efficient Deep Reinforcement Learning for Dialog Control

2016-12-18 · Kavosh Asadi, Jason D. Williams

Representing a dialog policy as a recurrent neural network (RNN) is attractive because it handles partial observability, infers a latent representation of state, and can be optimized with supervised learning (SL) or reinforcement learning (RL). For RL, a policy gradient approach is natural, but is sample inefficient. In this paper, we present 3 methods for reducing the number of dialogs required to optimize an RNN-based dialog policy with RL. The key idea is to maintain a second RNN which predicts the value of the current policy, and to apply experience replay to both networks. On two tasks, these methods reduce the number of dialogs/episodes required by about a third, vs. standard policy gradient methods.

📄 PDF Abstract BibTeX arXiv:1612.06000

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningPolicy Gradient Methodsreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management

2017-07-01 · WS 2017 8 · Pei-Hao Su, Pawel Budzianowski, Stefan Ultes, Milica Gasic 외

Deep reinforcement learning (RL) methods have significant potential for dialogue policy optimisation. However, they suffer from a poor performance in the early stages of learning. This is especially problematic for on-li…

Deep Reinforcement Learning Dialogue Management Management reinforcement-learning+2

SimpleDS: A Simple Deep Reinforcement Learning Dialogue System

2016-01-18 · Heriberto Cuayáhuitl

This paper presents 'SimpleDS', a simple and publicly available dialogue system trained with deep reinforcement learning. In contrast to previous reinforcement learning dialogue systems, this system avoids manual feature…

Deep Reinforcement Learning Feature Engineering reinforcement-learning Reinforcement Learning+1

DiactTOD: Learning Generalizable Latent Dialogue Acts for Controllable Task-Oriented Dialogue Systems

2023-08-01 · Qingyang Wu, James Gung, Raphael Shu, Yi Zhang

Dialogue act annotations are important to improve response generation quality in task-oriented dialogue systems. However, it can be challenging to use dialogue acts to control response generation in a generalizable way b…

Response Generation Task-Oriented Dialogue Systems

Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning

2017-02-10 · ACL 2017 7 · Jason D. Williams, Kavosh Asadi, Geoffrey Zweig

End-to-end learning of recurrent neural networks (RNNs) is an attractive solution for dialog systems; however, current techniques are data-intensive and require thousands of dialogs to learn simple behaviors. We introduc…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management

2021-04-10 · NAACL 2021 4 · Zhengxu Hou, Bang Liu, Ruihui Zhao, Zijing Ou 외

For task-oriented dialog systems, training a Reinforcement Learning (RL) based Dialog Management module suffers from low sample efficiency and slow convergence speed due to the sparse rewards in RL.To solve this problem,…

Management reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)