Multi-agent Reinforcement Learning for Energy Saving in Multi-Cell Massive MIMO Systems

2024-02-05 · Tianzhang Cai, Qichen Wang, Shuai Zhang, Özlem Tuğfe Demir, Cicek Cavdar

We develop a multi-agent reinforcement learning (MARL) algorithm to minimize the total energy consumption of multiple massive MIMO (multiple-input multiple-output) base stations (BSs) in a multi-cell network while preserving the overall quality-of-service (QoS) by making decisions on the multi-level advanced sleep modes (ASMs) and antenna switching of these BSs. The problem is modeled as a decentralized partially observable Markov decision process (DEC-POMDP) to enable collaboration between individual BSs, which is necessary to tackle inter-cell interference. A multi-agent proximal policy optimization (MAPPO) algorithm is designed to learn a collaborative BS control policy. To enhance its scalability, a modified version called MAPPO-neighbor policy is further proposed. Simulation results demonstrate that the trained MAPPO agent achieves better performance compared to baseline policies. Specifically, compared to the auto sleep mode 1 (symbol-level sleeping) algorithm, the MAPPO-neighbor policy reduces power consumption by approximately 8.7% during low-traffic hours and improves energy efficiency by approximately 19% during high-traffic hours, respectively.

📄 PDF Abstract BibTeX arXiv:2402.03204

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning

Methods 이 논문이 사용한 방법론

BASE 설명 없음

Energy Saving for Cell-Free Massive MIMO Networks: A Multi-Agent Deep Reinforcement Learning Approach

2026-04-08 · Qichen Wang, Keyu Li, Ozan Alp Topal, Özlem Tugfe Demir 외 arxiv

This paper focuses on energy savings in downlink operation of cell-free massive MIMO (CF mMIMO) networks under dynamic traffic conditions. We propose a multi-agent deep reinforcement learning (MADRL) algorithm that enabl…

Reinforcement Learning

Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

2021-10-26 · Vinay Hanumaiah, Sahika Genc

It is estimated that about 40%-50% of total electricity consumption in commercial buildings can be attributed to Heating, Ventilation, and Air Conditioning (HVAC) systems. Minimizing the energy cost while considering the…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning

2021-11-21 · Daniel J. B. Harrold, Jun Cao, Zhong Fan

In this paper, multi-agent reinforcement learning is used to control a hybrid energy storage system working collaboratively to reduce the energy costs of a microgrid through maximising the value of renewable energy and t…

Deep Reinforcement Learning energy trading Multi-agent Reinforcement Learning reinforcement-learning+1

Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning

2022-09-16 · William Wong, Praneet Dutta, Octavian Voicu, Yuri Chervonyi 외

Reinforcement learning (RL) techniques have been developed to optimize industrial cooling systems, offering substantial energy savings compared to traditional heuristic policies. A major challenge in industrial control i…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning a Multi-Agent Controller for Shared Energy Storage System

2023-02-16 · Ruohong Liu, Yize Chen

Deployment of shared energy storage systems (SESS) allows users to use the stored energy to meet their own energy demands while saving energy costs without installing private energy storage equipment. In this paper, we c…

Multi-agent Reinforcement Learning Scheduling