State-based Episodic Memory for Multi-Agent Reinforcement Learning

2021-10-19 · Xiao Ma, Wu-Jun Li

Multi-agent reinforcement learning (MARL) algorithms have made promising progress in recent years by leveraging the centralized training and decentralized execution (CTDE) paradigm. However, existing MARL algorithms still suffer from the sample inefficiency problem. In this paper, we propose a simple yet effective approach, called state-based episodic memory (SEM), to improve sample efficiency in MARL. SEM adopts episodic memory (EM) to supervise the centralized training procedure of CTDE in MARL. To the best of our knowledge, SEM is the first work to introduce EM into MARL. We can theoretically prove that, when using for MARL, SEM has lower space complexity and time complexity than state and action based EM (SAEM), which is originally proposed for single-agent reinforcement learning. Experimental results on StarCraft multi-agent challenge (SMAC) show that introducing episodic memory into MARL can improve sample efficiency and SEM can reduce storage cost and time cost compared with SAEM.

📄 PDF Abstract BibTeX arXiv:2110.09817

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)SMAC SMAC+Starcraft

Two-Memory Reinforcement Learning

2023-04-20 · Zhao Yang, Thomas. M. Moerland, Mike Preuss, Aske Plaat

While deep reinforcement learning has shown important empirical success, it tends to learn relatively slow due to slow propagation of rewards information and slow update of parametric neural networks. Non-parametric epis…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Representation Learning+1

Efficient Replay Memory Architectures in Multi-Agent Reinforcement Learning for Traffic Congestion Control

2024-07-22 · Mukul Chodhary, Kevin Octavian, SooJean Han

Episodic control, inspired by the role of episodic memory in the human brain, has been shown to improve the sample inefficiency of model-free reinforcement learning by reusing high-return past experiences. However, the m…

Decision Making Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Now I Remember! Episodic Memory For Reinforcement Learning

2018-01-01 · ICLR 2018 1 · Ricky Loynd, Matthew Hausknecht, Lihong Li, Li Deng

Humans rely on episodic memory constantly, in remembering the name of someone they met 10 minutes ago, the plot of a movie as it unfolds, or where they parked the car. Endowing reinforcement learning agents with episodic…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learning

2024-03-02 · Hyungho Na, Yunkyeong Seo, Il-Chul Moon

In cooperative multi-agent reinforcement learning (MARL), agents aim to achieve a common goal, such as defeating enemies or scoring a goal. Existing MARL algorithms are effective but still require significant learning ti…

Decoder Multi-agent Reinforcement Learning Q-Learning reinforcement-learning+3

Model-Based Episodic Memory Induces Dynamic Hybrid Controls

2021-11-03 · NeurIPS 2021 12 · Hung Le, Thommen Karimpanal George, Majid Abdolshah, Truyen Tran 외

Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory. We propose a new model-based episodic memory of trajectories addressing current limitations of e…

model reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)