PIC: Permutation Invariant Critic for Multi-Agent Deep Reinforcement Learning

2019-10-31 · Iou-Jen Liu, Raymond A. Yeh, Alexander G. Schwing

Sample efficiency and scalability to a large number of agents are two important goals for multi-agent reinforcement learning systems. Recent works got us closer to those goals, addressing non-stationarity of the environment from a single agent's perspective by utilizing a deep net critic which depends on all observations and actions. The critic input concatenates agent observations and actions in a user-specified order. However, since deep nets aren't permutation invariant, a permuted input changes the critic output despite the environment remaining identical. To avoid this inefficiency, we propose a 'permutation invariant critic' (PIC), which yields identical output irrespective of the agent permutation. This consistent representation enables our model to scale to 30 times more agents and to achieve improvements of test episode reward between 15% to 50% on the challenging multi-agent particle environment (MPE).

📄 PDF Abstract BibTeX arXiv:1911.00025

Code (2)

IouJenLiu/PIC 공식 구현 pytorch

zowiezhang/stas pytorch

Tasks

Deep Reinforcement LearningMulti-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Test 설명 없음

HARP: Human-Assisted Regrouping with Permutation Invariant Critic for Multi-Agent Reinforcement Learning

2024-09-18 · Huawen Hu, Enze Shi, Chenxi Yue, Shuocun Yang 외

Human-in-the-loop reinforcement learning integrates human expertise to accelerate agent learning and provide critical guidance and feedback in complex fields. However, many existing approaches focus on single-agent tasks…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

A Principled Permutation Invariant Approach to Mean-Field Multi-Agent Reinforcement Learning

2021-09-29 · Yan Li, Lingxiao Wang, Jiachen Yang, Ethan Wang 외

Multi-agent reinforcement learning (MARL) becomes more challenging in the presence of more agents, as the capacity of the joint state and action spaces grows exponentially in the number of agents. To address such a chall…

Inductive Bias Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Permutation Invariant Policy Optimization for Mean-Field Multi-Agent Reinforcement Learning: A Principled Approach

2021-05-18 · Yan Li, Lingxiao Wang, Jiachen Yang, Ethan Wang 외

Inductive Bias Multi-agent Reinforcement Learning

Hierarchical Reinforcement Learning for Optimal Agent Grouping in Cooperative Systems

2025-01-11 · Liyuan Hu

This paper presents a hierarchical reinforcement learning (RL) approach to address the agent grouping or pairing problem in cooperative multi-agent systems. The goal is to simultaneously learn the optimal grouping and ag…

Decision Making Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

SplAgger: Split Aggregation for Meta-Reinforcement Learning

2024-03-05 · Jacob Beck, Matthew Jackson, Risto Vuorio, Zheng Xiong 외

A core ambition of reinforcement learning (RL) is the creation of agents capable of rapid learning in novel tasks. Meta-RL aims to achieve this by directly learning such agents. Black box methods do so by training off-th…

continuous-control Continuous Control Meta Reinforcement Learning reinforcement-learning+2