A transformer-based deep reinforcement learning approach to spatial navigation in a partially observable Morris Water Maze

2024-10-01 · Marte Eggen, Inga Strümke

Navigation is a fundamental cognitive skill extensively studied in neuroscientific experiments and has lately gained substantial interest in artificial intelligence research. Recreating the task solved by rodents in the well-established Morris Water Maze (MWM) experiment, this work applies a transformer-based architecture using deep reinforcement learning -- an approach previously unexplored in this context -- to navigate a 2D version of the maze. Specifically, the agent leverages a decoder-only transformer architecture serving as a deep Q-network performing effective decision making in the partially observable environment. We demonstrate that the proposed architecture enables the agent to efficiently learn spatial navigation strategies, overcoming challenges associated with a limited field of vision, corresponding to the visual information available to a rodent in the MWM. Demonstrating the potential of transformer-based models for enhancing navigation performance in partially observable environments, this work suggests promising avenues for future research in artificial agents whose behavior resembles that of biological agents. Finally, the flexibility of the transformer architecture in supporting varying input sequence lengths opens opportunities for gaining increased understanding of the artificial agent's inner representation of the environment.

📄 PDF Abstract BibTeX arXiv:2410.12820

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making DecoderDeep Reinforcement LearningNavigate

A Reinforcement Learning-Based Model for Mapping and Goal-Directed Navigation Using Multiscale Place Fields

2026-01-07 · Bekarys Dukenbaev, Andrew Gerstenslager, Alexander Johnson, Ali A. Minai arxiv

Autonomous navigation in complex and partially observable environments remains a central challenge in robotics. Several bio-inspired models of mapping and navigation based on place cells in the mammalian hippocampus have…

Reinforcement Learning Robot Navigation

Sequence-level Intrinsic Exploration Model for Partially Observable Domains

2019-09-25 · Haiyan Yin, Jianda Chen, Sinno Jialin Pan

Training reinforcement learning policies in partially observable domains with sparse reward signal is an important and open problem for the research community. In this paper, we introduce a new sequence-level intrinsic n…

Prediction reinforcement-learning Reinforcement Learning (RL)

SGN-CIRL: Scene Graph-based Navigation with Curriculum, Imitation, and Reinforcement Learning

2025-06-04 · Nikita Oskolkov, Huzhenyu Zhang, Dmitry Makarov, Dmitry Yudin 외

The 3D scene graph models spatial relationships between objects, enabling the agent to efficiently navigate in a partially observable environment and predict the location of the target object.This paper proposes an origi…

Imitation Learning Navigate reinforcement-learning Reinforcement Learning+1

Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning

2020-11-09 · Shuijing Liu, Peixin Chang, Weihang Liang, Neeloy Chakraborty 외

Safe and efficient navigation through human crowds is an essential capability for mobile robots. Previous work on robot crowd navigation assumes that the dynamics of all agents are known and well-defined. In addition, th…

Deep Reinforcement Learning Motion Planning reinforcement-learning Reinforcement Learning (RL)+2

POPGym: Benchmarking Partially Observable Reinforcement Learning

2023-03-03 · Steven Morad, Ryan Kortvelesy, Matteo Bettini, Stephan Liwicki 외

Real world applications of Reinforcement Learning (RL) are often partially observable, thus requiring memory. Despite this, partial observability is still largely ignored by contemporary RL benchmarks and libraries. We i…

Benchmarking GPU Partially Observable Reinforcement Learning reinforcement-learning+4