UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning

2021-11-22 · Christopher Diehl, Timo Sievernich, Martin Krüger, Frank Hoffmann, Torsten Bertram

Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data and therefore constitutes a promising approach for real-world applications as automated driving. Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set. Especially in safety-critical applications as automated driving, explainability and transferability are key to success. This motivates the use of model-based offline RL approaches, which leverage planning. However, current state-of-the-art methods often neglect the influence of aleatoric uncertainty arising from the stochastic behavior of multi-agent systems. This work proposes a novel approach for Uncertainty-aware Model-Based Offline REinforcement Learning Leveraging plAnning (UMBRELLA), which solves the prediction, planning, and control problem of the SDV jointly in an interpretable learning-based fashion. A trained action-conditioned stochastic dynamics model captures distinctively different future evolutions of the traffic scene. The analysis provides empirical evidence for the effectiveness of our approach in challenging automated driving simulations and based on a real-world public dataset.

📄 PDF Abstract BibTeX arXiv:2111.11097

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Uncertainty-Aware Rank-One MIMO Q Network Framework for Accelerated Offline Reinforcement Learning

2026-02-23 · Thanh Nguyen, Tung Luu, Tri Ton, Sungwoong Kim 외 arxiv

Offline reinforcement learning (RL) has garnered significant interest due to its safe and easily scalable paradigm. However, training under this paradigm presents its own challenge: the extrapolation error stemming from …

Reinforcement Learning Offline RL

Uncertainty-aware Distributional Offline Reinforcement Learning

2024-03-26 · Xiaocong Chen, Siyu Wang, Tong Yu, Lina Yao

Offline reinforcement learning (RL) presents distinct challenges as it relies solely on observational data. A central concern in this context is ensuring the safety of the learned policy by quantifying uncertainties asso…

Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

SAMBO-RL: Shifts-aware Model-based Offline Reinforcement Learning

2024-08-23 · Wang Luo, Haoran Li, ZiCheng Zhang, Congying Han 외

Model-based Offline Reinforcement Learning trains policies based on offline datasets and model dynamics, without direct real-world environment interactions. However, this method is inherently challenged by distribution s…

reinforcement-learning Reinforcement Learning

Efficient and Uncertainty-Aware Diffusion Framework for Offline-to-Online Reinforcement Learning

2026-05-29 · Ha Manh Bui, Metod Jazbec, Eric Nalisnick, Anqi Liu arxiv

Offline-to-Online Reinforcement Learning (O2O-RL) leverages an offline, pre-trained policy to minimize costly online interactions. Although data-efficient, O2O-RL is susceptible to shifts between offline and online distr…

Reinforcement Learning

Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness

2023-09-29 · Xiaoyu Wen, Xudong Yu, Rui Yang, HaoYuan Chen 외

To obtain a near-optimal policy with fewer interactions in Reinforcement Learning (RL), a promising approach involves the combination of offline RL, which enhances sample efficiency by leveraging offline datasets, and on…

Offline RL reinforcement-learning Reinforcement Learning (RL)