Remember and Forget Experience Replay for Multi-Agent Reinforcement Learning

2022-03-24 · Pascal Weber, Daniel Wälchli, Mustafa Zeqiri, Petros Koumoutsakos

We present the extension of the Remember and Forget for Experience Replay (ReF-ER) algorithm to Multi-Agent Reinforcement Learning (MARL). ReF-ER was shown to outperform state of the art algorithms for continuous control in problems ranging from the OpenAI Gym to complex fluid flows. In MARL, the dependencies between the agents are included in the state-value estimator and the environment dynamics are modeled via the importance weights used by ReF-ER. In collaborative environments, we find the best performance when the value is estimated using individual rewards and we ignore the effects of other actions on the transition map. We benchmark the performance of ReF-ER MARL on the Stanford Intelligent Systems Laboratory (SISL) environments. We find that employing a single feed-forward neural network for the policy and the value function in ReF-ER MARL, outperforms state of the art algorithms that rely on complex neural network architectures.

📄 PDF Abstract BibTeX arXiv:2203.13319

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Multi-agent Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Remember and Forget for Experience Replay

2018-07-16 · ICLR 2019 5 · Guido Novati, Petros Koumoutsakos

Experience replay (ER) is a fundamental component of off-policy deep reinforcement learning (RL). ER recalls experiences from past iterations to compute gradient estimates for the current policy, increasing data-efficien…

Deep Reinforcement Learning Policy Gradient Methods Q-Learning Reinforcement Learning+1

Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes

2020-07-01 · Timothée Lesort

Humans learn all their life long. They accumulate knowledge from a sequence of learning experiences and remember the essential concepts without forgetting what they have learned previously. Artificial neural networks str…

Continual Learning

Prioritized Experience Replay

2015-11-18 · Tom Schaul, John Quan, Ioannis Antonoglou, David Silver

Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply r…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimization

2023-02-21 · Yongsheng Mei, Hanhan Zhou, Tian Lan, Guru Venkataramani 외

Experience replay is crucial for off-policy reinforcement learning (RL) methods. By remembering and reusing the experiences from past different policies, experience replay significantly improves the training efficiency a…

Decision Making Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+2

Prioritized Sequence Experience Replay

2019-05-25 · Marc Brittain, Josh Bertram, Xuxi Yang, Peng Wei

Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized …

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1