A Risk-Averse Preview-based $Q$-Learning Algorithm: Application to Highway Driving of Autonomous Vehicles
A risk-averse preview-based $Q$-learning planner is presented for navigation of autonomous vehicles. To this end, the multi-lane road ahead of a vehicle is represented by a finite-state non-stationary Markov decision process (MDP). A risk assessment unit module is then presented that leverages the preview information provided by sensors along with a stochastic reachability module to assign reward values to the MDP states and update them as scenarios develop. A sampling-based risk-averse preview-based Q-learning algorithm is finally developed that generates samples using the preview information and reward function to learn risk-averse optimal planning strategies without actual interaction with the environment. The risk factor is imposed on the objective function to avoid fluctuation of the Q values, which can jeopardize the vehicle's safety and/or performance. The overall hybrid automaton model of the system is leveraged to develop a feasibility check unit module that detects unfeasible plans and enables the planner system to react proactively to the changes of the environment. Finally, to verify the efficiency of the presented algorithm, its implementation on two highway driving scenarios of an autonomous vehicle in a varying traffic density is considered.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesQ-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Risk-Averse Approximate Dynamic Programming with Quantile-Based Risk Measures
In this paper, we consider a finite-horizon Markov decision process (MDP) for which the objective at each stage is to minimize a quantile-based risk measure (QBRM) of the sequence of future costs; we call the overall obj…
Risk-Averse Model Predictive Control for Priced Timed Automata
In this paper, we propose a Risk-Averse Priced Timed Automata (PTA) Model Predictive Control (MPC) framework to increase flexibility of cyber-physical systems. To improve flexibility in these systems, our risk-averse fra…
Decision MakingModel Predictive ControlRisk-Averse Offline Reinforcement Learning
Training Reinforcement Learning (RL) agents in high-stakes applications might be too prohibitive due to the risk associated to exploration. Thus, the agent can only use data previously collected by safe policies. While p…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Risk-averse learning with delayed feedback
In real-world scenarios, the impacts of decisions may not manifest immediately. Taking these delays into account facilitates accurate assessment and management of risk in real-world environments, thereby ensuring the eff…
ManagementRisk-Averse Stochastic Convex Bandit
Motivated by applications in clinical trials and finance, we study the problem of online convex optimization (with bandit feedback) where the decision maker is risk-averse. We provide two algorithms to solve this problem…