Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility

2022-03-07 · Flora Charbonnier, Thomas Morstyn, Malcolm D. McCulloch

This paper proposes a novel scalable type of multi-agent reinforcement learning-based coordination for distributed residential energy. Cooperating agents learn to control the flexibility offered by electric vehicles, space heating and flexible loads in a partially observable stochastic environment. In the standard independent Q-learning approach, the coordination performance of agents under partial observability drops at scale in stochastic environments. Here, the novel combination of learning from off-line convex optimisations on historical data and isolating marginal contributions to total rewards in reward signals increases stability and performance at scale. Using fixed-size Q-tables, prosumers are able to assess their marginal impact on total system objectives without sharing personal data either with each other or with a central coordinator. Case studies are used to assess the fitness of different combinations of exploration sources, reward definitions, and multi-agent learning frameworks. It is demonstrated that the proposed strategies create value at individual and system levels thanks to reductions in the costs of energy imports, losses, distribution network congestion, battery depreciation and greenhouse gas emissions.

📄 PDF Abstract BibTeX arXiv:2203.03417

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

TinyQMIX: Distributed Access Control for mMTC via Multi-agent Reinforcement Learning

2022-11-21 · Tien Thanh Le, Yusheng Ji, John C. S Lui

Distributed access control is a crucial component for massive machine type communication (mMTC). In this communication scenario, centralized resource allocation is not scalable because resource configurations have to be …

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systems

2022-12-13 · Xin Liu, Honghao Wei, Lei Ying

This paper studies a class of multi-agent reinforcement learning (MARL) problems where the reward that an agent receives depends on the states of other agents, but the next state only depends on the agent's own current s…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Centralised rehearsal of decentralised cooperation: Multi-agent reinforcement learning for the scalable coordination of residential energy flexibility

2023-05-30 · Flora Charbonnier, Bei Peng, Thomas Morstyn, Malcolm McCulloch

This paper investigates how deep multi-agent reinforcement learning can enable the scalable and privacy-preserving coordination of residential energy flexibility. The coordination of distributed resources such as electri…

Multi-agent Reinforcement Learning Privacy Preserving reinforcement-learning Reinforcement Learning

Scalable Reinforcement Learning for Linear-Quadratic Control of Networks

2024-01-29 · Johan Olsson, Runyu Zhang, Emma Tegling, Na Li

Distributed optimal control is known to be challenging and can become intractable even for linear-quadratic regulator problems. In this work, we study a special class of such problems where distributed state feedback con…

reinforcement-learning Reinforcement Learning

The AI Arena: A Framework for Distributed Multi-Agent Reinforcement Learning

2021-03-09 · Edward W. Staley, Corban G. Rivera, Ashley J. Llorens

Advances in reinforcement learning (RL) have resulted in recent breakthroughs in the application of artificial intelligence (AI) across many different domains. An emerging landscape of development environments is making …

Multi-agent Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning+1