Chance-Constrained Control with Lexicographic Deep Reinforcement Learning

2020-10-19 · Alessandro Giuseppi, Antonio Pietrabissa

This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes, in which the controller seeks to ensure that the probability of satisfying the constraint is above a given threshold. Standard DeepRL approaches require i) the constraints to be included as additional weighted terms in the cost function, in a multi-objective fashion, and ii) the tuning of the introduced weights during the training phase of the Deep Neural Network (DNN) according to the probability thresholds. The proposed approach, instead, requires to separately train one constraint-free DNN and one DNN associated to each constraint and then, at each time-step, to select which DNN to use depending on the system observed state. The presented solution does not require any hyper-parameter tuning besides the standard DNN ones, even if the probability thresholds changes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations.

📄 PDF Abstract BibTeX arXiv:2010.09468

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Lexicographic Multi-Objective Reinforcement Learning

2022-12-28 · Joar Skalse, Lewis Hammond, Charlie Griffin, Alessandro Abate

In this work we introduce reinforcement learning techniques for solving lexicographic multi-objective problems. These are problems that involve multiple reward signals, and where the goal is to learn a policy that maximi…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning

2021-02-17 · Baiyu Peng, Yao Mu, Jingliang Duan, Yang Guan 외

Safety is essential for reinforcement learning (RL) applied in real-world tasks like autonomous driving. Chance constraints which guarantee the satisfaction of state constraints at a high probability are suitable to repr…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Model-based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangian

2021-08-26 · Baiyu Peng, Jingliang Duan, Jianyu Chen, Shengbo Eben Li 외

Safety is essential for reinforcement learning (RL) applied in the real world. Adding chance constraints (or probabilistic constraints) is a suitable way to enhance RL safety under uncertainty. Existing chance-constraine…

reinforcement-learning Reinforcement Learning (RL)Robot Navigation

Safe Chance Constrained Reinforcement Learning for Batch Process Control

2021-04-23 · Max Mowbray, Panagiotis Petsagkourakis, Ehecatl Antonio del Río Chanona, Dongda Zhang

Reinforcement Learning (RL) controllers have generated excitement within the control community. The primary advantage of RL controllers relative to existing methods is their ability to optimize uncertain systems independ…

Gaussian Processes Model Predictive Control reinforcement-learning Reinforcement Learning+1

Model-Based Actor-Critic with Chance Constraint for Stochastic System

2020-12-19 · Baiyu Peng, Yao Mu, Yang Guan, Shengbo Eben Li 외

Safety is essential for reinforcement learning (RL) applied in real-world situations. Chance constraints are suitable to represent the safety requirements in stochastic systems. Previous chance-constrained RL methods usu…

Model Predictive Control Reinforcement Learning (RL)