Diluted Near-Optimal Expert Demonstrations for Guiding Dialogue Stochastic Policy Optimisation

2020-11-25 · Thibault Cordier, Tanguy Urvoy, Lina M. Rojas-Barahona, Fabrice Lefèvre

A learning dialogue agent can infer its behaviour from interactions with the users. These interactions can be taken from either human-to-human or human-machine conversations. However, human interactions are scarce and costly, making learning from few interactions essential. One solution to speedup the learning process is to guide the agent's exploration with the help of an expert. We present in this paper several imitation learning strategies for dialogue policy where the guiding expert is a near-optimal handcrafted policy. We incorporate these strategies with state-of-the-art reinforcement learning methods based on Q-learning and actor-critic. We notably propose a randomised exploration policy which allows for a seamless hybridisation of the learned policy and the expert. Our experiments show that our hybridisation strategy outperforms several baselines, and that it can accelerate the learning when facing real humans.

📄 PDF Abstract BibTeX arXiv:2012.04687

Code (0)

등록된 구현이 없습니다.

Tasks

Imitation LearningQ-LearningReinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Self-Imitation Learning from Demonstrations

2022-03-21 · Georgiy Pshikhachev, Dmitry Ivanov, Vladimir Egorov, Aleksei Shpilman

Despite the numerous breakthroughs achieved with Reinforcement Learning (RL), solving environments with sparse rewards remains a challenging task that requires sophisticated exploration. Learning from Demonstrations (LfD…

Imitation Learning Reinforcement Learning (RL)

Teaching Inverse Reinforcement Learners via Features and Demonstrations

2018-10-21 · NeurIPS 2018 12 · Luis Haug, Sebastian Tschiatschek, Adish Singla

Learning near-optimal behaviour from an expert's demonstrations typically relies on the assumption that the learner knows the features that the true reward function depends on. In this paper, we study the problem of lear…

Reinforcement Learning

Learning to control from expert demonstrations

2022-03-09 · Alimzhan Sultangazin, Luigi Pannocchi, Lucas Fraile, Paulo Tabuada

In this paper, we revisit the problem of learning a stabilizing controller from a finite number of demonstrations by an expert. By first focusing on feedback linearizable systems, we show how to combine expert demonstrat…

A Reduction from Apprenticeship Learning to Classification

2010-12-01 · NeurIPS 2010 12 · Umar Syed, Robert E. Schapire

We provide new theoretical results for apprenticeship learning, a variant of reinforcement learning in which the true reward function is unknown, and the goal is to perform well relative to an observed expert. We study a…

Classification General Classification Reinforcement Learning

Learning Sparse Rewarded Tasks from Sub-Optimal Demonstrations

2020-04-01 · Zhuangdi Zhu, Kaixiang Lin, Bo Dai, Jiayu Zhou

Model-free deep reinforcement learning (RL) has demonstrated its superiority on many complex sequential decision-making problems. However, heavy dependence on dense rewards and high sample-complexity impedes the wide ado…

continuous-control Continuous Control Decision Making Deep Reinforcement Learning+4