Structure-Enhanced DRL for Optimal Transmission Scheduling
Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, we focus on the transmission scheduling problem of a remote estimation system. First, we derive some structural properties of the optimal sensor scheduling policy over fading channels. Then, building on these theoretical guidelines, we develop a structure-enhanced deep reinforcement learning (DRL) framework for optimal scheduling of the system to achieve the minimum overall estimation mean-square error (MSE). In particular, we propose a structure-enhanced action selection method, which tends to select actions that obey the policy structure. This explores the action space more effectively and enhances the learning efficiency of DRL agents. Furthermore, we introduce a structure-enhanced loss function to add penalties to actions that do not follow the policy structure. The new loss function guides the DRL to converge to the optimal policy structure quickly. Our numerical experiments illustrate that the proposed structure-enhanced DRL algorithms can save the training time by 50% and reduce the remote estimation MSE by 10% to 25% when compared to benchmark DRL algorithms. In addition, we show that the derived structural properties exist in a wide range of dynamic scheduling problems that go beyond remote state estimation.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningSchedulingState EstimationSimilar Papers 제목 키워드 기반
Structure-Enhanced Deep Reinforcement Learning for Optimal Transmission Scheduling
Remote state estimation of large-scale distributed dynamic processes plays an important role in Industry 4.0 applications. In this paper, by leveraging the theoretical results of structural properties of optimal scheduli…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)+2Optimal Transmission Power Scheduling for Networked Control System under DoS Attack
Designing networked control systems that are reliable and resilient against adversarial threats, is essential for ensuring the security of cyber-physical systems. This paper addresses the communication-control co-design …
SchedulingInfinite-horizon optimal scheduling for feedback control
Emerging cyber-physical systems impel the development of communication protocols that optimize resource utilization. This article investigates infinite-horizon optimal scheduling for resource-aware networked control syst…
SchedulingWireless Link Scheduling with State-Augmented Graph Neural Networks
We consider the problem of optimal link scheduling in large-scale wireless ad hoc networks. We specifically aim for the maximum long-term average performance, subject to a minimum transmission requirement for each link t…
FairnessGraph Neural NetworkSchedulingMinimum Length Scheduling for Multi-cell Full Duplex Wireless Powered Communication Networks
In this paper, we investigate a novel minimum length scheduling problem to determine the optimal power control, and scheduling for constant and continuous rate models, while considering concurrent transmission of users, …
Scheduling