Selective Reincarnation: Offline-to-Online Multi-Agent Reinforcement Learning

2023-03-31 · Claude Formanek, Callum Rhys Tilbury, Jonathan Shock, Kale-ab Tessera, Arnu Pretorius

'Reincarnation' in reinforcement learning has been proposed as a formalisation of reusing prior computation from past experiments when training an agent in an environment. In this paper, we present a brief foray into the paradigm of reincarnation in the multi-agent (MA) context. We consider the case where only some agents are reincarnated, whereas the others are trained from scratch -- selective reincarnation. In the fully-cooperative MA setting with heterogeneous agents, we demonstrate that selective reincarnation can lead to higher returns than training fully from scratch, and faster convergence than training with full reincarnation. However, the choice of which agents to reincarnate in a heterogeneous system is vitally important to the outcome of the training -- in fact, a poor choice can lead to considerably worse results than the alternatives. We argue that a rich field of work exists here, and we hope that our effort catalyses further energy in bringing the topic of reincarnation to the multi-agent realm.

📄 PDF Abstract BibTeX arXiv:2304.00977

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Offline Pre-trained Multi-Agent Decision Transformer

2021-09-29 · Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le 외

Offline reinforcement learning leverages static datasets to learn optimal policies with no necessity to access the environment. This is desirable for multi-agent systems due to the expensiveness of agents' online interac…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Offline-to-Online Multi-Agent Reinforcement Learning with Offline Value Function Memory and Sequential Exploration

2024-10-25 · Hai Zhong, Xun Wang, Zhuoran Li, Longbo Huang

Offline-to-Online Reinforcement Learning has emerged as a powerful paradigm, leveraging offline data for initialization and online fine-tuning to enhance both sample efficiency and performance. However, most existing res…

Efficient Exploration Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+3

Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability

2026-05-31 · Xianyou Li, Weiran Yan, Yichao Wu, Penghao Liang 외 arxiv

Failure-aware observability diagnoses wasted computation in multi-agent LLM systems before final-answer evaluation can explain what went wrong. We propose a trace-based framework for a three-agent architecture -- orchest…

Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Tasks

2021-12-06 · Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le 외

Offline reinforcement learning leverages previously-collected offline datasets to learn optimal policies with no necessity to access the real environment. Such a paradigm is also desirable for multi-agent reinforcement l…

All Multi-agent Reinforcement Learning Offline RL reinforcement-learning+5

Online Tuning for Offline Decentralized Multi-Agent Reinforcement Learning

2021-09-29 · Jiechuan Jiang, Zongqing Lu

Offline reinforcement learning could learn effective policies from a fixed dataset, which is promising in real-world applications. However, in offline decentralized multi-agent reinforcement learning, due to the discrepa…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)