Agent-Time Attention for Sparse Rewards Multi-Agent Reinforcement Learning

2022-10-31 · Jennifer She, Jayesh K. Gupta, Mykel J. Kochenderfer

Sparse and delayed rewards pose a challenge to single agent reinforcement learning. This challenge is amplified in multi-agent reinforcement learning (MARL) where credit assignment of these rewards needs to happen not only across time, but also across agents. We propose Agent-Time Attention (ATA), a neural network model with auxiliary losses for redistributing sparse and delayed rewards in collaborative MARL. We provide a simple example that demonstrates how providing agents with their own local redistributed rewards and shared global redistributed rewards motivate different policies. We extend several MiniGrid environments, specifically MultiRoom and DoorKey, to the multi-agent sparse delayed rewards setting. We demonstrate that ATA outperforms various baselines on many instances of these environments. Source code of the experiments is available at https://github.com/jshe/agent-time-attention.

📄 PDF Abstract BibTeX arXiv:2210.17540

Code (1)

jshe/agent-time-attention 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Agent-Temporal Credit Assignment for Optimal Policy Preservation in Sparse Multi-Agent Reinforcement Learning

2024-12-19 · Aditya Kapoor, Sushant Swamy, Kale-ab Tessera, Mayank Baranwal 외

In multi-agent environments, agents often struggle to learn optimal policies due to sparse or delayed global rewards, particularly in long-horizon tasks where it is challenging to evaluate actions at intermediate time st…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning TAR

AC-MASAC: An Attentive Curriculum Learning Framework for Heterogeneous UAV Swarm Coordination

2026-02-12 · Wanhao Liu, Junhong Dai, Yixuan Zhang, Shengyun Yin 외 arxiv

Cooperative path planning for heterogeneous UAV swarms poses significant challenges for Multi-Agent Reinforcement Learning (MARL), particularly in handling asymmetric inter-agent dependencies and addressing the risks of …

Multi-agent Reinforcement Learning

Action Guidance: Getting the Best of Sparse Rewards and Shaped Rewards for Real-time Strategy Games

2020-10-05 · Shengyi Huang, Santiago Ontañón

Training agents using Reinforcement Learning in games with sparse rewards is a challenging problem, since large amounts of exploration are required to retrieve even the first reward. To tackle this problem, a common appr…

Real-Time Strategy Games Reinforcement Learning (RL)

Dealing with Sparse Rewards Using Graph Neural Networks

2022-03-25 · Matvey Gerasyov, Ilya Makarov

Deep reinforcement learning in partially observable environments is a difficult task in itself, and can be further complicated by a sparse reward signal. Most tasks involving navigation in three-dimensional environments …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Shaping Advice in Deep Multi-Agent Reinforcement Learning

2021-03-29 · Baicen Xiao, Bhaskar Ramasubramanian, Radha Poovendran

Multi-agent reinforcement learning involves multiple agents interacting with each other and a shared environment to complete tasks. When rewards provided by the environment are sparse, agents may not receive immediate fe…