Multi-Agent Reinforcement Learning with Multi-Step Generative Models

2019-01-29 · Orr Krupnik, Igor Mordatch, Aviv Tamar

We consider model-based reinforcement learning (MBRL) in 2-agent, high-fidelity continuous control problems -- an important domain for robots interacting with other agents in the same workspace. For non-trivial dynamical systems, MBRL typically suffers from accumulating errors. Several recent studies have addressed this problem by learning latent variable models for trajectory segments and optimizing over behavior in the latent space. In this work, we investigate whether this approach can be extended to 2-agent competitive and cooperative settings. The fundamental challenge is how to learn models that capture interactions between agents, yet are disentangled to allow for optimization of each agent behavior separately. We propose such models based on a disentangled variational auto-encoder, and demonstrate our approach on a simulated 2-robot manipulation task, where one robot can either help or distract the other. We show that our approach has better sample efficiency than a strong model-free RL baseline, and can learn both cooperative and adversarial behavior from the same data.

📄 PDF Abstract BibTeX arXiv:1901.10251

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Decision MakingModel-based Reinforcement LearningMulti-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Robot Manipulation

OM2P: Offline Multi-Agent Mean-Flow Policy

2025-08-08 · Zhuoran Li, Xun Wang, Hai Zhong, Qingxin Xia 외 arxiv

Generative models, especially diffusion and flow-based models, have been promising in offline multi-agent reinforcement learning. However, integrating powerful generative models into this framework poses unique challenge…

Multi-agent Reinforcement Learning

Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination

2024-03-05 · Liangzhou Wang, Kaiwen Zhu, Fengming Zhu, Xinghu Yao 외

Reaching consensus is key to multi-agent coordination. To accomplish a cooperative task, agents need to coherently select optimal joint actions to maximize the team reward. However, current cooperative multi-agent reinfo…

Multi-agent Reinforcement Learning

Multiscale Generative Models: Improving Performance of a Generative Model Using Feedback from Other Dependent Generative Models

2022-01-24 · Changyu Chen, Avinandan Bose, Shih-Fen Cheng, Arunesh Sinha

Realistic fine-grained multi-agent simulation of real-world complex systems is crucial for many downstream tasks such as reinforcement learning. Recent work has used generative models (GANs in particular) for providing h…

Time Series Time Series Analysis Transfer Learning

Distributed Policy Iteration for Scalable Approximation of Cooperative Multi-Agent Policies

2019-01-25 · Thomy Phan, Kyrill Schmid, Lenz Belzner, Thomas Gabor 외

Decision making in multi-agent systems (MAS) is a great challenge due to enormous state and joint action spaces as well as uncertainty, making centralized control generally infeasible. Decentralized control offers better…

Decision Making Multi-agent Reinforcement Learning Reinforcement Learning

Shaping Belief States with Generative Environment Models for RL

2019-06-21 · NeurIPS 2019 12 · Karol Gregor, Danilo Jimenez Rezende, Frederic Besse, Yan Wu 외

When agents interact with a complex environment, they must form and maintain beliefs about the relevant aspects of that environment. We propose a way to efficiently train expressive generative models in complex environme…

Reinforcement Learning Reinforcement Learning (RL)