Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions

2020-07-05 · Michael Chang, Sidhant Kaushik, S. Matthew Weinberg, Thomas L. Griffiths, Sergey Levine

This paper seeks to establish a framework for directing a society of simple, specialized, self-interested agents to solve what traditionally are posed as monolithic single-agent sequential decision problems. What makes it challenging to use a decentralized approach to collectively optimize a central objective is the difficulty in characterizing the equilibrium strategy profile of non-cooperative games. To overcome this challenge, we design a mechanism for defining the learning environment of each agent for which we know that the optimal solution for the global objective coincides with a Nash equilibrium strategy profile of the agents optimizing their own local objectives. The society functions as an economy of agents that learn the credit assignment process itself by buying and selling to each other the right to operate on the environment state. We derive a class of decentralized reinforcement learning algorithms that are broadly applicable not only to standard reinforcement learning but also for selecting options in semi-MDPs and dynamically composing computation graphs. Lastly, we demonstrate the potential advantages of a society's inherent modular structure for more efficient transfer learning.

📄 PDF Abstract BibTeX arXiv:2007.02382

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learning

2025-09-20 · Wei Duan, Jie Lu, Junyu Xuan arxiv

In networked multi-agent reinforcement learning (Networked-MARL), decentralized agents must act under local observability and constrained communication over fixed physical graphs. Existing methods often assume static nei…

Multi-agent Reinforcement Learning

Decentralized Deep Reinforcement Learning for Delay-Power Tradeoff in Vehicular Communications

2019-06-03 · Xianfu Chen, Celimuge Wu, Honggang Zhang, Yan Zhang 외

This paper targets at the problem of radio resource management for expected long-term delay-power tradeoff in vehicular communications. At each decision epoch, the road side unit observes the global network state, alloca…

Decision Making Deep Reinforcement Learning Management reinforcement-learning+3

Decentralized Cross-Entropy Method for Model-Based Reinforcement Learning

2021-09-29 · Zichen Zhang, Jun Jin, Martin Jagersand, Jun Luo 외

Cross-Entropy Method (CEM) is a popular approach to planning in model-based reinforcement learning. It has so far always taken a \textit{centralized} approach where the sampling distribution is updated \textit{centrally}…

continuous-control Continuous Control Decision Making Model-based Reinforcement Learning+4

Learn How to Query from Unlabeled Data Streams in Federated Learning

2024-12-11 · Yuchang Sun, Xinran Li, Tao Lin, Jun Zhang

Federated learning (FL) enables collaborative learning among decentralized clients while safeguarding the privacy of their local data. Existing studies on FL typically assume offline labeled data available at each client…

Benchmarking Decision Making Federated Learning Multi-agent Reinforcement Learning

Dimension-Free Rates for Natural Policy Gradient in Multi-Agent Reinforcement Learning

2021-09-23 · Carlo Alfano, Patrick Rebeschini

Cooperative multi-agent reinforcement learning is a decentralized paradigm in sequential decision making where agents distributed over a network iteratively collaborate with neighbors to maximize global (network-wide) no…

Decision Making Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)+1