Model-based Multi-Agent Reinforcement Learning with Cooperative Prioritized Sweeping

2020-01-15 · Eugenio Bargiacchi, Timothy Verstraeten, Diederik M. Roijers, Ann Nowé

We present a new model-based reinforcement learning algorithm, Cooperative Prioritized Sweeping, for efficient learning in multi-agent Markov decision processes. The algorithm allows for sample-efficient learning on large problems by exploiting a factorization to approximate the value function. Our approach only requires knowledge about the structure of the problem in the form of a dynamic decision network. Using this information, our method learns a model of the environment and performs temporal difference updates which affect multiple joint states and actions at once. Batch updates are additionally performed which efficiently back-propagate knowledge throughout the factored Q-function. Our method outperforms the state-of-the-art algorithm sparse cooperative Q-learning algorithm, both on the well-known SysAdmin benchmark and randomized environments.

📄 PDF Abstract BibTeX arXiv:2001.07527

Code (0)

등록된 구현이 없습니다.

Tasks

Model-based Reinforcement LearningMulti-agent Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Prioritized Sweeping Prioritized Sweeping is a reinforcement learning technique for model-based algorithms that prioritizes updates according to a measure of urgency, and performs these updates…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

2018-02-15 · Lise Aubin, Mehdi Khamassi, Benoît Girard

During sleep and awake rest, the hippocampus replays sequences of place cells that have been activated during prior experiences. These have been interpreted as a memory consolidation process, but recent results suggest a…

Hippocampus Q-Learning reinforcement-learning Reinforcement Learning+1

Is prioritized sweeping the better episodic control?

2017-11-20 · Johanni Brea

Episodic control has been proposed as a third approach to reinforcement learning, besides model-free and model-based control, by analogy with the three types of human memory. i.e. episodic, procedural and semantic memory…

Reinforcement Learning

Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping

2012-06-13 · Richard S. Sutton, Csaba Szepesvari, Alborz Geramifard, Michael P. Bowling

We consider the problem of efficiently learning optimal control policies and value functions over large state spaces in an online setting in which estimates must be available after each interaction with the world. This p…

Experience Sharing Between Cooperative Reinforcement Learning Agents

2019-11-06 · Lucas Oliveira Souza, Gabriel de Oliveira Ramos, Celia Ghedini Ralha

The idea of experience sharing between cooperative agents naturally emerges from our understanding of how humans learn. Our evolution as a species is tightly linked to the ability to exchange learned knowledge with one a…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Prioritized Guidance for Efficient Multi-Agent Reinforcement Learning Exploration

2019-07-18 · Qisheng Wang, Qichao Wang

Exploration efficiency is a challenging problem in multi-agent reinforcement learning (MARL), as the policy learned by confederate MARL depends on the collaborative approach among multiple agents. Another important probl…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)