Causality Detection for Efficient Multi-Agent Reinforcement Learning

2023-03-24 · Rafael Pina, Varuna De Silva, Corentin Artaud

When learning a task as a team, some agents in Multi-Agent Reinforcement Learning (MARL) may fail to understand their true impact in the performance of the team. Such agents end up learning sub-optimal policies, demonstrating undesired lazy behaviours. To investigate this problem, we start by formalising the use of temporal causality applied to MARL problems. We then show how causality can be used to penalise such lazy agents and improve their behaviours. By understanding how their local observations are causally related to the team reward, each agent in the team can adjust their individual credit based on whether they helped to cause the reward or not. We show empirically that using causality estimations in MARL improves not only the holistic performance of the team, but also the individual capabilities of each agent. We observe that the improvements are consistent in a set of different environments.

📄 PDF Abstract BibTeX arXiv:2303.14227

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

fail 설명 없음

Discovering Causality for Efficient Cooperation in Multi-Agent Environments

2023-06-20 · Rafael Pina, Varuna De Silva, Corentin Artaud

In cooperative Multi-Agent Reinforcement Learning (MARL) agents are required to learn behaviours as a team to achieve a common goal. However, while learning a task, some agents may end up learning sub-optimal policies, n…

Causal Discovery Multi-agent Reinforcement Learning

Causal Multi-Agent Reinforcement Learning: Review and Open Problems

2021-11-12 · St John Grimbly, Jonathan Shock, Arnu Pretorius

This paper serves to introduce the reader to the field of multi-agent reinforcement learning (MARL) and its intersection with methods from the study of causality. We highlight key challenges in MARL and discuss these in …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Causal Mean Field Multi-Agent Reinforcement Learning

2025-02-20 · Hao Ma, Zhiqiang Pu, Yi Pan, Boyin Liu 외

Scalability remains a challenge in multi-agent reinforcement learning and is currently under active research. A framework named mean-field reinforcement learning (MFRL) could alleviate the scalability problem by employin…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning

Object-Centric World Models for Causality-Aware Reinforcement Learning

2025-11-18 · Yosuke Nishimoto, Takashi Matsubara arxiv

World models have been developed to support sample-efficient deep reinforcement learning agents. However, it remains challenging for world models to accurately replicate environments that are high-dimensional, non-statio…

Reinforcement Learning

Learning Independently from Causality in Multi-Agent Environments

2023-11-05 · Rafael Pina, Varuna De Silva, Corentin Artaud

Multi-Agent Reinforcement Learning (MARL) comprises an area of growing interest in the field of machine learning. Despite notable advances, there are still problems that require investigation. The lazy agent pathology is…

Multi-agent Reinforcement Learning Relation