Age-of-information minimization under energy harvesting and non-stationary environment
This work focuses on minimizing the age of information for multiple energy harvesting sources that sample data and transmit it to a sink node. At each time, the central scheduler selects one of the sources to probe the quality of its channel to the sink node, and then the assessed channel quality is utilized to determine whether a source will sample and send the packet. For a single source case, we assume that the probed channel quality is known at each time instant, model the problem of AoI minimization as a Markov decision process, and prove the optimal sampling policy threshold structure. We then use this threshold structure and propose an AEC-SW-UCRL2 algorithm to handle unknown and time varying energy harvesting rate and channel statistics, motivated by the popular SWUCRL2 algorithm for non stationary reinforcement learning. This algorithm is applicable when an upper bound is available for the total variation of each of these quantities over a time horizon. Furthermore, in situations where these variation budgets are not accessible, we introduce the AEC-BORL algorithm, motivated by the well known BORL algorithm. For the multiple source case, we demonstrate that the AoI minimization problem can be formulated as a constrained MDP, which can be relaxed using a Lagrange multiplier and decoupled into sub problems across source nodes. We also derive Whittle index based source scheduling policy for probing and an optimal threshold policy for source sampling. We next leverage this Whittle index and threshold structure to develop the WIT-SW-UCRL2 algorithm for unknown time varying energy harvesting rates and channel statistics under their respective variation budgets. Moreover, we also proposed a Whittle index and threshold based bandit over reinforcement learning (WIT-BORL) algorithm for unknown variation budgets. Finally, we numerically demonstrate the efficacy of our algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
SchedulingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Average AoI Minimization for Energy Harvesting Relay-aided Status Update Network Using Deep Reinforcement Learning
A dual-hop status update system aided by energy harvesting (EH) relays with finite data and energy buffers is studied in this work. To achieve timely status updates, the best relays should be selected to minimize the ave…
Deep Reinforcement LearningAge-of-information minimization via opportunistic sampling by an energy harvesting source
Herein, minimization of time-averaged age-of-information (AoI) in an energy harvesting (EH) source setting is considered. The EH source opportunistically samples one or multiple processes over discrete time instants and …
Q-LearningEnergy Efficient UAV Communication with Energy Harvesting
This paper investigates an unmanned aerial vehicle (UAV)-enabled wireless communication system with energy harvesting, where the UAV transfers energy to the users in half duplex or full duplex, and the users harvest ener…
AoI Minimization in Energy Harvesting and Spectrum Sharing Enabled 6G Networks
Spectrum sharing is a method to solve the problem of frequency spectrum deficiency. This paper studies a novel AI based spectrum sharing and energy harvesting system in which the freshness of information (AoI) is guarant…
Whittle's index-based age-of-information minimization in multi-energy harvesting source networks
We consider the problem of source sampling and transmission scheduling for age-of-information minimization in a system consisting of multiple energy harvesting (EH) sources and a sink node. At each time, one of the sourc…
Q-LearningScheduling