Memory-Efficient Episodic Control Reinforcement Learning with Dynamic Online k-means
Recently, neuro-inspired episodic control (EC) methods have been developed to overcome the data-inefficiency of standard deep reinforcement learning approaches. Using non-/semi-parametric models to estimate the value function, they learn rapidly, retrieving cached values from similar past states. In realistic scenarios, with limited resources and noisy data, maintaining meaningful representations in memory is essential to speed up the learning and avoid catastrophic forgetting. Unfortunately, EC methods have a large space and time complexity. We investigate different solutions to these problems based on prioritising and ranking stored states, as well as online clustering techniques. We also propose a new dynamic online k-means algorithm that is both computationally-efficient and yields significantly better performance at smaller memory sizes; we validate this approach on classic reinforcement learning environments and Atari games.
Code (1)
Tasks
Atari GamesClusteringDeep Reinforcement LearningOnline Clusteringreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Model-Based Episodic Memory Induces Dynamic Hybrid Controls
Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory. We propose a new model-based episodic memory of trajectories addressing current limitations of e…
modelreinforcement-learningReinforcement LearningReinforcement Learning (RL)Solving Continuous Control with Episodic Memory
Episodic memory lets reinforcement learning algorithms remember and exploit promising experience from the past to improve agent performance. Previous works on memory mechanisms show benefits of using episodic-based data …
continuous-controlContinuous ControlOpenAI GymReinforcement Learning (RL)Efficient Replay Memory Architectures in Multi-Agent Reinforcement Learning for Traffic Congestion Control
Episodic control, inspired by the role of episodic memory in the human brain, has been shown to improve the sample inefficiency of model-free reinforcement learning by reusing high-return past experiences. However, the m…
Decision MakingMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+1Sequential memory improves sample and memory efficiency in Episodic Control
State of the art deep reinforcement learning algorithms are sample inefficient due to the large number of episodes they require to achieve asymptotic performance. Episodic Reinforcement Learning (ERL) algorithms, inspire…
Deep Reinforcement LearningHippocampusreinforcement-learningReinforcement Learning+1Continuous Episodic Control
Non-parametric episodic memory can be used to quickly latch onto high-rewarded experience in reinforcement learning tasks. In contrast to parametric deep reinforcement learning approaches in which reward signals need to …
continuous-controlContinuous ControlDecision MakingDeep Reinforcement Learning+4