Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

2021-09-26 · Gaoyang Pang, Kang Huang, Daniel E. Quevedo, Branka Vucetic, Yonghui Li, Wanchun Liu

We consider a joint uplink and downlink scheduling problem of a fully distributed wireless networked control system (WNCS) with a limited number of frequency channels. Using elements of stochastic systems theory, we derive a sufficient stability condition of the WNCS, which is stated in terms of both the control and communication system parameters. Once the condition is satisfied, there exists a stationary and deterministic scheduling policy that can stabilize all plants of the WNCS. By analyzing and representing the per-step cost function of the WNCS in terms of a finite-length countable vector state, we formulate the optimal transmission scheduling problem into a Markov decision process and develop a deep reinforcement learning (DRL) based framework for solving it. To tackle the challenges of a large action space in DRL, we propose novel action space reduction and action embedding methods for the DRL framework that can be applied to various algorithms, including Deep Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed Deep Deterministic Policy Gradient (TD3). Numerical results show that the proposed algorithm significantly outperforms benchmark policies.

📄 PDF Abstract BibTeX arXiv:2109.12562

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)Scheduling

Fully Distributed Online Training of Graph Neural Networks in Networked Systems

2024-12-08 · Rostyslav Olshevskyi, Zhongyuan Zhao, Kevin Chan, Gunjan Verma 외

Graph neural networks (GNNs) are powerful tools for developing scalable, decentralized artificial intelligence in large-scale networked systems, such as wireless networks, power grids, and transportation networks. Curren…

Node Regression Scheduling

Resource Optimization for Tail-Based Control in Wireless Networked Control Systems

2024-06-20 · Rasika Vijithasena, Rafaela Scaciota, Mehdi Bennis, Sumudu Samarakoon

Achieving control stability is one of the key design challenges of scalable Wireless Networked Control Systems (WNCS) under limited communication and computing resources. This paper explores the use of an alternative con…

GPR Prediction Reinforcement Learning (RL)Scheduling

Communication-Control Codesign for Large-Scale Wireless Networked Control Systems

2024-10-15 · Gaoyang Pang, Wanchun Liu, Dusit Niyato, Branka Vucetic 외

Wireless Networked Control Systems (WNCSs) are essential to Industry 4.0, enabling flexible control in applications, such as drone swarms and autonomous robots. The interdependence between communication and control requi…

Deep Reinforcement Learning Scheduling Sequential Decision Making

Deep reinforcement learning for scheduling in large-scale networked control systems

2019-05-15 · Adrian Redder, Arunselvan Ramaswamy, Daniel E. Quevedo

This work considers the problem of control and resource scheduling in networked systems. We present DIRA, a Deep reinforcement learning based Iterative Resource Allocation algorithm, which is scalable and control-aware. …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach

2022-10-03 · Zihuai Zhao, Wanchun Liu, Daniel E. Quevedo, Yonghui Li 외

Wireless networked control system (WNCS) connecting sensors, controllers, and actuators via wireless communications is a key enabling technology for highly scalable and low-cost deployment of control systems in the Indus…

Deep Reinforcement Learning Scheduling