Efficient Dialogue Complementary Policy Learning via Deep Q-network Policy and Episodic Memory Policy
Deep reinforcement learning has shown great potential in training dialogue policies. However, its favorable performance comes at the cost of many rounds of interaction. Most of the existing dialogue policy methods rely on a single learning system, while the human brain has two specialized learning and memory systems, supporting to find good solutions without requiring copious examples. Inspired by the human brain, this paper proposes a novel complementary policy learning (CPL) framework, which exploits the complementary advantages of the episodic memory (EM) policy and the deep Q-network (DQN) policy to achieve fast and effective dialogue policy learning. In order to coordinate between the two policies, we proposed a confidence controller to control the complementary time according to their relative efficacy at different stages. Furthermore, memory connectivity and time pruning are proposed to guarantee the flexible and adaptive generalization of the EM policy in dialog tasks. Experimental results on three dialogue datasets show that our method significantly outperforms existing methods relying on a single learning system.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Episodic Policy Gradient Training
We introduce a novel training procedure for policy gradient methods wherein episodic memory is used to optimize the hyperparameters of reinforcement learning algorithms on-the-fly. Unlike other hyperparameter searches, w…
Policy Gradient MethodsSchedulingSolving Continuous Control with Episodic Memory
Episodic memory lets reinforcement learning algorithms remember and exploit promising experience from the past to improve agent performance. Previous works on memory mechanisms show benefits of using episodic-based data …
continuous-controlContinuous ControlOpenAI GymReinforcement Learning (RL)Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
Episodic memory plays an important role in the behavior of animals and humans. It allows the accumulation of information about current state of the environment in a task-agnostic way. This episodic representation can be …
Continual Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Memento 2: Learning by Stateful Reflective Memory
We present a theoretical study of continual and experiential learning in large language model agents that combine episodic memory with reinforcement learning. We argue that the key mechanism for continual adaptation, wit…
Reinforcement LearningTwo-Memory Reinforcement Learning
While deep reinforcement learning has shown important empirical success, it tends to learn relatively slow due to slow propagation of rewards information and slow update of parametric neural networks. Non-parametric epis…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningRepresentation Learning+1