LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning

2019-12-01 · NeurIPS 2019 12 · Yali Du, Lei Han, Meng Fang, Ji Liu, Tianhong Dai, DaCheng Tao

A great challenge in cooperative decentralized multi-agent reinforcement learning (MARL) is generating diversified behaviors for each individual agent when receiving only a team reward. Prior studies have paid much effort on reward shaping or designing a centralized critic that can discriminatively credit the agents. In this paper, we propose to merge the two directions and learn each agent an intrinsic reward function which diversely stimulates the agents at each time step. Specifically, the intrinsic reward for a specific agent will be involved in computing a distinct proxy critic for the agent to direct the updating of its individual policy. Meanwhile, the parameterized intrinsic reward function will be updated towards maximizing the expected accumulated team reward from the environment so that the objective is consistent with the original MARL problem. The proposed method is referred to as learning individual intrinsic reward (LIIR) in MARL. We compare LIIR with a number of state-of-the-art MARL methods on battle games in StarCraft II. The results demonstrate the effectiveness of LIIR, and we show LIIR can assign each individual agent an insightful intrinsic reward per time step.

📄 PDF Abstract BibTeX

Code (1)

yalidu/liir 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)StarcraftStarcraft II

The Emergence of Individuality in Multi-Agent Reinforcement Learning

2020-09-28 · Jiechuan Jiang, Zongqing Lu

Individuality is essential in human society, which induces the division of labor and thus improves the efficiency and productivity. Similarly, it should also be a key to multi-agent cooperation. Inspired by that individu…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

The Emergence of Individuality

2020-06-10 · Jiechuan Jiang, Zongqing Lu

Individuality is essential in human society, which induces the division of labor and thus improves the efficiency and productivity. Similarly, it should also be the key to multi-agent cooperation. Inspired by that indivi…

Multi-agent Reinforcement Learning

Curiosity-Driven Multi-Agent Exploration with Mixed Objectives

2022-10-29 · Roben Delos Reyes, Kyunghwan Son, Jinhwan Jung, Wan Ju Kang 외

Intrinsic rewards have been increasingly used to mitigate the sparse reward problem in single-agent reinforcement learning. These intrinsic rewards encourage the agent to look for novel experiences, guiding the agent to …

Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning via Incorporating Generalized Human Expertise

2025-07-25 · Xuefei Wu, Xiao Yin, Yuanyang Zhu, Chunlin Chen arxiv

Efficient exploration in multi-agent reinforcement learning (MARL) is a challenging problem when receiving only a team reward, especially in environments with sparse rewards. A powerful method to mitigate this issue invo…

Multi-agent Reinforcement Learning

AIIR-MIX: Multi-Agent Reinforcement Learning Meets Attention Individual Intrinsic Reward Mixing Network

2023-02-19 · Wei Li, Weiyan Liu, Shitong Shao, Shiyi Huang

Deducing the contribution of each agent and assigning the corresponding reward to them is a crucial problem in cooperative Multi-Agent Reinforcement Learning (MARL). Previous studies try to resolve the issue through desi…

Multi-agent Reinforcement Learning Starcraft Starcraft II