AoI Minimization in Status Update Control with Energy Harvesting Sensors
Information freshness is crucial for time-critical IoT applications, e.g., monitoring and control systems. We consider an IoT status update system with multiple users, multiple energy harvesting sensors, and a wireless edge node. The users receive time-sensitive information about physical quantities, each measured by a sensor. Users send requests to the edge node where a cache contains the most recently received measurements from each sensor. To serve a request, the edge node either commands the sensor to send a status update or retrieves the aged measurement from the cache. We aim at finding the best actions of the edge node to minimize the age of information of the served measurements. We model this problem as a Markov decision process and develop reinforcement learning (RL) algorithms: model-based value iteration and model-free Q-learning methods. We also propose a Q-learning method for the realistic case where the edge node is informed about the sensors' battery levels only via the status updates. The case under transmission limitations is also addressed. Furthermore, properties of an optimal policy are analytically characterized. Simulation results show that an optimal policy is a threshold-based policy and that the proposed RL methods significantly reduce the average cost compared to several baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Q-LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Average AoI Minimization for Energy Harvesting Relay-aided Status Update Network Using Deep Reinforcement Learning
A dual-hop status update system aided by energy harvesting (EH) relays with finite data and energy buffers is studied in this work. To achieve timely status updates, the best relays should be selected to minimize the ave…
Deep Reinforcement LearningLearning to Minimize Age of Information over an Unreliable Channel with Energy Harvesting
The time average expected age of information (AoI) is studied for status updates sent over an error-prone channel from an energy-harvesting transmitter with a finite-capacity battery. Energy cost of sensing new status up…
Reinforcement Learning (RL)SchedulingAge-Aware Status Update Control for Energy Harvesting IoT Sensors via Reinforcement Learning
We consider an IoT sensing network with multiple users, multiple energy harvesting sensors, and a wireless edge node acting as a gateway between the users and sensors. The users request for updates about the value of phy…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Status Updating under Partial Battery Knowledge in Energy Harvesting IoT Networks
We study status updating under inexact knowledge about the battery levels of the energy harvesting sensors in an IoT network, where users make on-demand requests to a cache-enabled edge node to send updates about various…
Decision MakingOn the Age-Optimality of Relax-then-Truncate Approach under Partial Battery Knowledge in Energy Harvesting IoT Networks
We consider an energy harvesting (EH) IoT network, where users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an EH sensor. The edge node ser…