Learning Reward Machines in Cooperative Multi-Agent Tasks

2023-03-24 · Leo Ardon, Daniel Furelos-Blanco, Alessandra Russo

This paper presents a novel approach to Multi-Agent Reinforcement Learning (MARL) that combines cooperative task decomposition with the learning of reward machines (RMs) encoding the structure of the sub-tasks. The proposed method helps deal with the non-Markovian nature of the rewards in partially observable environments and improves the interpretability of the learnt policies required to complete the cooperative task. The RMs associated with each sub-task are learnt in a decentralised manner and then used to guide the behaviour of each agent. By doing so, the complexity of a cooperative multi-agent problem is reduced, allowing for more effective learning. The results suggest that our approach is a promising direction for future research in MARL, especially in complex environments with large state spaces and multiple agents.

📄 PDF Abstract BibTeX arXiv:2303.14061

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning

Reward Machines for Cooperative Multi-Agent Reinforcement Learning

2020-07-03 · Cyrus Neary, Zhe Xu, Bo Wu, Ufuk Topcu

In cooperative multi-agent reinforcement learning, a collection of agents learns to interact in a shared environment to achieve a common goal. We propose the use of reward machines (RM) -- Mealy machines used as structur…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

Multi-Agent Reinforcement Learning with a Hierarchy of Reward Machines

2024-03-08 · Xuejing Zheng, Chao Yu

In this paper, we study the cooperative Multi-Agent Reinforcement Learning (MARL) problems using Reward Machines (RMs) to specify the reward functions such that the prior knowledge of high-level events in a task can be l…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARL

2022-08-02 · Jakub Grudzien Kuba, Xidong Feng, Shiyao Ding, Hao Dong 외

The necessity for cooperation among intelligent machines has popularised cooperative multi-agent reinforcement learning (MARL) in the artificial intelligence (AI) research community. However, many research endeavors have…

MuJoCo Multi-agent Reinforcement Learning

Inducing Cooperation via Learning to reshape rewards in semi-cooperative multi-agent reinforcement learning

2019-05-01 · ICLR 2019 5 · David Earl Hostallero, Daewoo Kim, Kyunghwan Son, Yung Yi

We propose a deep reinforcement learning algorithm for semi-cooperative multi-agent tasks, where agents are equipped with their separate reward functions, yet with willingness to cooperate. Under these semi-cooperative s…

Deep Reinforcement Learning Multi-agent Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

Learning Symbolic Task Decompositions for Multi-Agent Teams

2025-02-19 · Ameesh Shah, Niklas Lauffer, Thomas Chen, Nikhil Pitta 외

One approach for improving sample efficiency in cooperative multi-agent learning is to decompose overall tasks into sub-tasks that can be assigned to individual agents. We study this problem in the context of reward mach…

Deep Reinforcement Learning