Provable Reinforcement Learning for Networked Control Systems with Stochastic Packet Disordering

2023-12-05 · Wenqian Xue, Yi Jiang, Frank L. Lewis, Bosen Lian

This paper formulates a stochastic optimal control problem for linear networked control systems featuring stochastic packet disordering with a unique stabilizing solution certified. The problem is solved by proposing reinforcement learning algorithms. A measurement method is first presented to deal with PD and calculate the newest control input. The NCSs with stochastic PD are modeled as stochastic NCSs. Then, given a cost function, a modified algebraic Riccati equation is derived within the formulation. We propose offline policy iteration and value iteration algorithms to solve the MARE associated with provable convergence. These two algorithms require knowledge of NCS dynamics and PD probabilities. To release that, we further design online model-free off-policy and Q-learning algorithms with an online estimation method for PD probability. Both model-free algorithms solve the optimal control problem using real-time system states, control inputs, and PD probability estimates. Simulation results verify the proposed formulation and algorithms at last.

📄 PDF Abstract BibTeX arXiv:2312.02498

Code (0)

등록된 구현이 없습니다.

Tasks

Q-Learningreinforcement-learning

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Stability and $\mathcal{H}_{\infty}$ Performance Analysis of Stochastic Linear Networked and Quantized Control Systems

2021-03-04 · Wei Ren, Junlin Xiong

This paper studies the stability and $\mathcal{H}_{\infty}$ performance analysis problem for linear networked and quantized control systems with both communication delays random packet losses. To deal with the network-in…

Deep reinforcement learning for scheduling in large-scale networked control systems

2019-05-15 · Adrian Redder, Arunselvan Ramaswamy, Daniel E. Quevedo

This work considers the problem of control and resource scheduling in networked systems. We present DIRA, a Deep reinforcement learning based Iterative Resource Allocation algorithm, which is scalable and control-aware. …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Provable Reinforcement Learning for Networked Control Systems with Stochastic Packet Disordering

Code (0)

Tasks

Methods 이 논문이 사용한 방법론

Similar Papers 제목 키워드 기반

Stability and $\mathcal{H}_{\infty}$ Performance Analysis of Stochastic Linear Networked and Quantized Control Systems

Deep reinforcement learning for scheduling in large-scale networked control systems

Safety Embedded Stochastic Optimal Control of Networked Multi-Agent Systems via Barrier States

Stochastic Aperiodic Control of Networked Systems with i.i.d. Time-Varying Communication Delays

Multi-Agent Reinforcement Learning in Stochastic Networked Systems