On Memory Mechanism in Multi-Agent Reinforcement Learning

2019-09-11 · Yilun Zhou, Derrik E. Asher, Nicholas R. Waytowich, Julie A. Shah

Multi-agent reinforcement learning (MARL) extends (single-agent) reinforcement learning (RL) by introducing additional agents and (potentially) partial observability of the environment. Consequently, algorithms for solving MARL problems incorporate various extensions beyond traditional RL methods, such as a learned communication protocol between cooperative agents that enables exchange of private information or adaptive modeling of opponents in competitive settings. One popular algorithmic construct is a memory mechanism such that an agent's decisions can depend not only upon the current state but also upon the history of observed states and actions. In this paper, we study how a memory mechanism can be useful in environments with different properties, such as observability, internality and presence of a communication channel. Using both prior work and new experiments, we show that a memory mechanism is helpful when learning agents need to model other agents and/or when communication is constrained in some way; however we must to be cautious of agents achieving effective memoryfulness through other means.

📄 PDF Abstract BibTeX arXiv:1909.05232

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Semantic HELM: A Human-Readable Memory for Reinforcement Learning

2023-06-15 · NeurIPS 2023 11 · Fabian Paischer, Thomas Adler, Markus Hofmarcher, Sepp Hochreiter

Reinforcement learning agents deployed in the real world often have to cope with partially observable environments. Therefore, most agents employ memory mechanisms to approximate the state of the environment. Recently, t…

Dota 2 Language Modelling Minecraft reinforcement-learning+3

Integrating Episodic Memory into a Reinforcement Learning Agent using Reservoir Sampling

2018-06-01 · ICLR 2018 1 · Kenny J. Young, Richard S. Sutton, Shuo Yang

Episodic memory is a psychology term which refers to the ability to recall specific events from the past. We suggest one advantage of this particular type of memory is the ability to easily assign credit to a specific st…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning

2025-08-27 · Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie 외 arxiv

Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of NLP tasks, but they remain fundamentally stateless, constrained by limited context windows that hinder long-horizon reasoning.…

Reinforcement Learning

Recurrent Action Transformer with Memory

2023-06-15 · Egor Cherepanov, Alexey Staroverov, Dmitry Yudin, Alexey K. Kovalev 외

Recently, the use of transformers in offline reinforcement learning has become a rapidly developing area. This is due to their ability to treat the agent's trajectory in the environment as a sequence, thereby reducing th…

Atari Games MuJoCo reinforcement-learning Reinforcement Learning

Interactive Memory Learning for Long-Term Conversations

2026-09-15 · Cai Ke, Jiangyue Yan, Han Zhang, Xin Liu 외 arxiv

Recent advancements in large language models have significantly enhanced the capabilities of agents in modeling long-term conversations. Despite these successes, existing approaches typically adopt a static heuristic par…

Reinforcement Learning Test-time Adaptation