Hierarchical Deep Reinforcement Learning for Age-of-Information Minimization in IRS-aided and Wireless-powered Wireless Networks
In this paper, we focus on a wireless-powered sensor network coordinated by a multi-antenna access point (AP). Each node can generate sensing information and report the latest information to the AP using the energy harvested from the AP's signal beamforming. We aim to minimize the average age-of-information (AoI) by adapting the nodes' transmission scheduling and the transmission control strategies jointly. To reduce the transmission delay, an intelligent reflecting surface (IRS) is used to enhance the channel conditions by controlling the AP's beamforming vector and the IRS's phase shifting matrix. Considering dynamic data arrivals at different sensing nodes, we propose a hierarchical deep reinforcement learning (DRL) framework to for AoI minimization in two steps. The users' transmission scheduling is firstly determined by the outer-loop DRL approach, e.g. the DQN or PPO algorithm, and then the inner-loop optimization is used to adapt either the uplink information transmission or downlink energy transfer to all nodes. A simple and efficient approximation is also proposed to reduce the inner-loop rum time overhead. Numerical results verify that the hierarchical learning framework outperforms typical baselines in terms of the average AoI and proportional fairness among different nodes.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningFairnessSchedulingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Survey on Model-based, Heuristic, and Machine Learning Optimization Approaches in RIS-aided Wireless Networks
Reconfigurable intelligent surfaces (RISs) have received considerable attention as a key enabler for envisioned 6G networks, for the purpose of improving the network capacity, coverage, efficiency, and security with low …
Federated LearningGraph LearningManagementTransfer LearningAge Minimization in Outdoor and Indoor Communications with Relay-aided Dual RIS
In this paper, we investigate an outdoor and indoor wireless communication network with the assistance of a novel relay-aided double-sided reconfigurable intelligent surface (RIS). A scheduling problem is considered at t…
SchedulingAverage AoI Minimization for Energy Harvesting Relay-aided Status Update Network Using Deep Reinforcement Learning
A dual-hop status update system aided by energy harvesting (EH) relays with finite data and energy buffers is studied in this work. To achieve timely status updates, the best relays should be selected to minimize the ave…
Deep Reinforcement LearningEnergy Minimization for Active RIS-Aided UAV-Enabled SWIPT Systems
In this paper, we consider an active reconfigurable intelligent surface (RIS)-aided unmanned aerial vehicle(UAV)-enabled simultaneous wireless information and power transfer(SWIPT) system with multiple ground users. Comp…
Cooperative Hierarchical Deep Reinforcement Learning based Joint Sleep and Power Control in RIS-aided Energy-Efficient RAN
Energy efficiency (EE) is one of the most important metrics for envisioned 6G networks, and sleep control, as a cost-efficient approach, can significantly lower power consumption by switching off network devices selectiv…
Decision MakingDeep Reinforcement Learning