Generalizable Episodic Memory for Deep Reinforcement Learning

2021-03-11 · Hao Hu, Jianing Ye, Guangxiang Zhu, Zhizhou Ren, Chongjie Zhang

Episodic memory-based methods can rapidly latch onto past successful strategies by a non-parametric memory and improve sample efficiency of traditional reinforcement learning. However, little effort is put into the continuous domain, where a state is never visited twice, and previous episodic methods fail to efficiently aggregate experience across trajectories. To address this problem, we propose Generalizable Episodic Memory (GEM), which effectively organizes the state-action values of episodic memory in a generalizable manner and supports implicit planning on memorized trajectories. GEM utilizes a double estimator to reduce the overestimation bias induced by value propagation in the planning process. Empirical evaluation shows that our method significantly outperforms existing trajectory-based methods on various MuJoCo continuous control tasks. To further show the general applicability, we evaluate our method on Atari games with discrete action space, which also shows a significant improvement over baseline algorithms.

📄 PDF Abstract BibTeX arXiv:2103.06469

Code (1)

MouseHu/GEM 공식 구현

Tasks

Atari Games continuous-control Continuous ControlDeep Reinforcement LearningMuJoCoreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Two-Memory Reinforcement Learning

2023-04-20 · Zhao Yang, Thomas. M. Moerland, Mike Preuss, Aske Plaat

While deep reinforcement learning has shown important empirical success, it tends to learn relatively slow due to slow propagation of rewards information and slow update of parametric neural networks. Non-parametric epis…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Representation Learning+1

Now I Remember! Episodic Memory For Reinforcement Learning

2018-01-01 · ICLR 2018 1 · Ricky Loynd, Matthew Hausknecht, Lihong Li, Li Deng

Humans rely on episodic memory constantly, in remembering the name of someone they met 10 minutes ago, the plot of a movie as it unfolds, or where they parked the car. Endowing reinforcement learning agents with episodic…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Flexible Prefrontal Control over Hippocampal Episodic Memory for Goal-Directed Generalization

2025-03-04 · Yicong Zheng, Nora Wolf, Charan Ranganath, Randall C. O'Reilly 외

Many tasks require flexibly modifying perception and behavior based on current goals. Humans can retrieve episodic memories from days to years ago, using them to contextualize and generalize behaviors across novel but st…

Hippocampus Retrieval

Model-Based Episodic Memory Induces Dynamic Hybrid Controls

2021-11-03 · NeurIPS 2021 12 · Hung Le, Thommen Karimpanal George, Majid Abdolshah, Truyen Tran 외

Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory. We propose a new model-based episodic memory of trajectories addressing current limitations of e…

model reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Episodic Reinforcement Learning with Associative Memory

2020-05-01 · ICLR 2020 1 · Guangxiang Zhu*, Zichuan Lin*, Guangwen Yang, Chongjie Zhang

Sample efficiency has been one of the major challenges for deep reinforcement learning. Non-parametric episodic control has been proposed to speed up parametric reinforcement learning by rapidly latching on previously su…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1