Multi-Agent Reinforcement Learning with a Hierarchy of Reward Machines

2024-03-08 · Xuejing Zheng, Chao Yu

In this paper, we study the cooperative Multi-Agent Reinforcement Learning (MARL) problems using Reward Machines (RMs) to specify the reward functions such that the prior knowledge of high-level events in a task can be leveraged to facilitate the learning efficiency. Unlike the existing work that RMs have been incorporated into MARL for task decomposition and policy learning in relatively simple domains or with an assumption of independencies among the agents, we present Multi-Agent Reinforcement Learning with a Hierarchy of RMs (MAHRM) that is capable of dealing with more complex scenarios when the events among agents can occur concurrently and the agents are highly interdependent. MAHRM exploits the relationship of high-level events to decompose a task into a hierarchy of simpler subtasks that are assigned to a small group of agents, so as to reduce the overall computational complexity. Experimental results in three cooperative MARL domains show that MAHRM outperforms other MARL methods using the same prior knowledge of high-level events.

📄 PDF Abstract BibTeX arXiv:2403.07005

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Hierarchies of Reward Machines

2022-05-31 · Daniel Furelos-Blanco, Mark Law, Anders Jonsson, Krysia Broda 외

Reward machines (RMs) are a recent formalism for representing the reward function of a reinforcement learning task through a finite-state machine whose edges encode subgoals of the task using high-level events. The struc…

Reinforcement Learning with Stochastic Reward Machines

2025-10-16 · Jan Corazza, Ivan Gavran, Daniel Neider arxiv

Reward machines are an established tool for dealing with reinforcement learning problems in which rewards are sparse and depend on complex sequences of actions. However, existing algorithms for learning reward machines a…

Reinforcement Learning

Automatic formation of the structure of abstract machines in hierarchical reinforcement learning with state clustering

2018-06-13 · Aleksandr I. Panov, Aleksey Skrynnik

We introduce a new approach to hierarchy formation and task decomposition in hierarchical reinforcement learning. Our method is based on the Hierarchy Of Abstract Machines (HAM) framework because HAM approach is able to …

Clustering Hierarchical Reinforcement Learning Q-Learning Reinforcement Learning

Inferring Probabilistic Reward Machines from Non-Markovian Reward Processes for Reinforcement Learning

2021-07-09 · Taylor Dohmen, Noah Topper, George Atia, Andre Beckus 외

The success of reinforcement learning in typical settings is predicated on Markovian assumptions on the reward signal by which an agent learns optimal policies. In recent years, the use of reward machines has relaxed thi…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning With Reward Machines in Stochastic Games

2023-05-27 · Jueming Hu, Jean-Raphael Gaglione, Yanze Wang, Zhe Xu 외

We investigate multi-agent reinforcement learning for stochastic games with complex tasks, where the reward functions are non-Markovian. We utilize reward machines to incorporate high-level knowledge of complex tasks. We…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning