Neural Map: Structured Memory for Deep Reinforcement Learning

2017-02-27 · ICLR 2018 1 · Emilio Parisotto, Ruslan Salakhutdinov

A critical component to enabling intelligent reasoning in partially observable environments is memory. Despite this importance, Deep Reinforcement Learning (DRL) agents have so far used relatively simple memory architectures, with the main methods to overcome partial observability being either a temporal convolution over the past k frames or an LSTM layer. More recent work (Oh et al., 2016) has went beyond these architectures by using memory networks which can allow more sophisticated addressing schemes over the past k frames. But even these architectures are unsatisfactory due to the reason that they are limited to only remembering information from the last k frames. In this paper, we develop a memory system with an adaptable write operator that is customized to the sorts of 3D environments that DRL agents typically interact with. This architecture, called the Neural Map, uses a spatially structured 2D memory image to learn to store arbitrary information about the environment over long time lags. We demonstrate empirically that the Neural Map surpasses previous DRL memories on a set of challenging 2D and 3D maze environments and show that it is capable of generalizing to environments that were not seen during training.

📄 PDF Abstract BibTeX arXiv:1702.08360

Code (1)

zuoxingdong/VIN_PyTorch_Visdom pytorch

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

EgoMap: Projective mapping and structured egocentric memory for Deep RL

2020-01-24 · Edward Beeching, Christian Wolf, Jilles Dibangoye, Olivier Simonin

Tasks involving localization, memorization and planning in partially observable 3D environments are an ongoing challenge in Deep Reinforcement Learning. We present EgoMap, a spatially structured neural memory architectur…

Deep Reinforcement Learning Memorization reinforcement-learning Reinforcement Learning+1

Memory Retention Is Not Enough to Master Memory Tasks in Reinforcement Learning

2026-01-21 · Oleg Shchendrigin, Egor Cherepanov, Alexey K. Kovalev, Aleksandr I. Panov arxiv

Effective decision-making in the real world depends on memory that is both stable and adaptive: environments change over time, and agents must retain relevant information over long horizons while also updating or overwri…

Reinforcement Learning

From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space

2026-07-07 · Yue Xu, Yutao Sun, Yihao Liu, Mengyu Zhou 외 arxiv

Long-term user memory is essential for personalized conversational agents, yet many memory systems still expose memory through passive retrieval interfaces, making the model a consumer of pre-selected evidence. We introd…

Reinforcement Learning

Explicit Recall for Efficient Exploration

2019-05-01 · ICLR 2019 5 · Honghua Dong, Jiayuan Mao, Xinyue Cui, Lihong Li

In this paper, we advocate the use of explicit memory for efficient exploration in reinforcement learning. This memory records structured trajectories that have led to interesting states in the past, and can be used by …

Decision Making Deep Reinforcement Learning Efficient Exploration reinforcement-learning+2

AllDayNav: Lifelong Navigation via Real-World Reinforcement Learning

2026-06-09 · Hang Yin, Yinan Liang, Jiazhao Zhang, Jiahang Liu 외 arxiv

Lifelong embodied navigation in dynamic environments requires robots to form persistent scene understanding from fragmentary observations, which remains difficult for existing methods that rely on explicit maps or scene …

Reinforcement Learning Scene Understanding