Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

2021-07-15 · Danial Kamran, Tizian Engelgeh, Marvin Busch, Johannes Fischer, Christoph Stiller

Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although punishing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies that can tune their level of conservativity at run-time based on the desired comfort and utility. Using a proactive safety verification approach, the proposed framework can guarantee that actions generated from RL are fail-safe according to the worst-case assumptions. Concurrently, the policy is encouraged to minimize safety interference and generate more comfortable behavior. We trained and evaluated the proposed approach and baseline policies using a high level simulator with a variety of randomized scenarios including several corner cases which rarely happen in reality but are very crucial. In light of our experiments, the behavior of policies learned using distributional RL can be adaptive at run-time and robust to the environment uncertainty. Quantitatively, the learned distributional RL agent drives in average 8 seconds faster than the normal DQN policy and requires 83\% less safety interference compared to the rule-based policy with slightly increasing the average crossing time. We also study sensitivity of the learned policy in environments with higher perception noise and show that our algorithm learns policies that can still drive reliable when the perception noise is two times higher than the training configuration for automated merging and crossing at occluded intersections.

📄 PDF Abstract BibTeX arXiv:2107.07316

Code (0)

등록된 구현이 없습니다.

Tasks

Autonomous VehiclesDistributional Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Hybrid Deep Reinforcement Learning and Planning for Safe and Comfortable Automated Driving

2022-12-30 · Dikshant Gupta, Mathias Klusch

We present a novel hybrid learning method, HyLEAR, for solving the collision-free navigation problem for self-driving cars in POMDPs. HyLEAR leverages interposed learning to embed knowledge of a hybrid planner into a dee…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Self-Driving Cars

High-level Decisions from a Safe Maneuver Catalog with Reinforcement Learning for Safe and Cooperative Automated Merging

2021-07-15 · Danial Kamran, Yu Ren, Martin Lauer

Reinforcement learning (RL) has recently been used for solving challenging decision-making problems in the context of automated driving. However, one of the main drawbacks of the presented RL-based policies is the lack o…

Decision Making Reinforcement Learning (RL)

Formal Certification Methods for Automated Vehicle Safety Assessment

2022-02-06 · Tong Zhao, Ekim Yurtsever, Joel Paulson, Giorgio Rizzoni

Challenges related to automated driving are no longer focused on just the construction of such automated vehicles (AVs), but in assuring the safety of their operation. Recent advances in Level 3 and Level 4 autonomous dr…

Autonomous Driving

MESA: Improving MoE Safety Alignment via Decentralized Expertise

2026-05-30 · Yitong Sun, Yao Huang, Teng Li, Ranjie Duan 외 arxiv

Mixture-of-Experts (MoE) architectures scale Large Language Models (LLMs) efficiently, enabling greater capacity with reduced computational cost by dynamically routing inputs to relevant experts, yet introduce a critical…

SECRM-2D: RL-Based Efficient and Comfortable Route-Following Autonomous Driving with Analytic Safety Guarantees

2024-07-23 · Tianyu Shi, Ilia Smirnov, Omar ElSamadisy, Baher Abdulhai

Over the last decade, there has been increasing interest in autonomous driving systems. Reinforcement Learning (RL) shows great promise for training autonomous driving controllers, being able to directly optimize a combi…

Autonomous Driving Reinforcement Learning (RL)