Cooperative Hierarchical Deep Reinforcement Learning based Joint Sleep and Power Control in RIS-aided Energy-Efficient RAN

2023-04-26 · Hao Zhou, Medhat Elsayed, Majid Bavand, Raimundas Gaigalas, Steve Furr, Melike Erol-Kantarci

Energy efficiency (EE) is one of the most important metrics for envisioned 6G networks, and sleep control, as a cost-efficient approach, can significantly lower power consumption by switching off network devices selectively. Meanwhile, the reconfigurable intelligent surface (RIS) has emerged as a promising technique to enhance the EE of future wireless networks. In this work, we jointly consider sleep and transmission power control for RIS-aided energy-efficient networks. In particular, considering the timescale difference between sleep control and power control, we introduce a cooperative hierarchical deep reinforcement learning (Co-HDRL) algorithm, enabling hierarchical and intelligent decision-making. Specifically, the meta-controller in Co-HDRL uses cross-entropy metrics to evaluate the policy stability of sub-controllers, and sub-controllers apply the correlated equilibrium to select optimal joint actions. Compared with conventional HDRL, Co-HDRL enables more stable high-level policy generations and low-level action selections. Then, we introduce a fractional programming method for RIS phase-shift control, maximizing the sum-rate under a given transmission power. In addition, we proposed a low-complexity surrogate optimization method as a baseline for RIS control. Finally, simulations show that the RIS-assisted sleep control can achieve more than 16\% lower energy consumption and 30\% higher EE than baseline algorithms.

📄 PDF Abstract BibTeX arXiv:2304.13226

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingDeep Reinforcement Learning

Hierarchical Reinforcement Learning for RIS-Assisted Energy-Efficient RAN

2023-01-07 · Hao Zhou, Long Kong, Medhat Elsayed, Majid Bavand 외

Reconfigurable intelligent surface (RIS) is emerging as a promising technology to boost the energy efficiency (EE) of 5G beyond and 6G networks. Inspired by this potential, in this paper, we investigate the RIS-assisted …

Hierarchical Reinforcement Learning Management reinforcement-learning Reinforcement Learning+1

Hierarchical Reinforcement Learning for Relay Selection and Power Optimization in Two-Hop Cooperative Relay Network

2020-11-10 · Yuanzhe Geng, Erwu Liu, Rui Wang, Yiming Liu

Cooperative communication is an effective approach to improve spectrum utilization. In order to reduce outage probability of communication system, most studies propose various schemes for relay selection and power alloca…

Deep Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

HCPO: Hierarchical Conductor-Based Policy Optimization in Multi-Agent Reinforcement Learning

2025-11-15 · Zejiao Liu, Junqi Tu, Yitian Hong, Luolin Xiong 외 arxiv

In cooperative Multi-Agent Reinforcement Learning (MARL), efficient exploration is crucial for optimizing the performance of joint policy. However, existing methods often update joint policies via independent agent explo…

Multi-agent Reinforcement Learning

Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey

2024-01-10 · Jiechuan Jiang, Kefan Su, Zongqing Lu

Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks, but restrictions of real-world applications may require training the agents in a fully decentralized manner. D…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Survey

Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensus

2026-04-15 · Zijian Zhao, Jing Gao, Sen Li arxiv

Cooperative multi-agent reinforcement learning (MARL) is widely used to address large joint observation and action spaces by decomposing a centralized control problem into multiple interacting agents. However, such decom…

Multi-agent Reinforcement Learning Decision Making Starcraft II