Practical Probabilistic Model-based Deep Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory Sampling

2023-09-20 · Wenjun Huang, Yunduan Cui, Huiyun Li, Xinyu Wu

This paper addresses the prediction stability, prediction accuracy and control capability of the current probabilistic model-based reinforcement learning (MBRL) built on neural networks. A novel approach dropout-based probabilistic ensembles with trajectory sampling (DPETS) is proposed where the system uncertainty is stably predicted by combining the Monte-Carlo dropout and trajectory sampling in one framework. Its loss function is designed to correct the fitting error of neural networks for more accurate prediction of probabilistic models. The state propagation in its policy is extended to filter the aleatoric uncertainty for superior control capability. Evaluated by several Mujoco benchmark control tasks under additional disturbances and one practical robot arm manipulation task, DPETS outperforms related MBRL approaches in both average return and convergence velocity while achieving superior performance than well-known model-free baselines with significant sample efficiency. The open source code of DPETS is available at https://github.com/mrjun123/DPETS.

📄 PDF Abstract BibTeX arXiv:2309.11089

Code (1)

mrjun123/DPETS 공식 구현 pytorch

Tasks

Deep Reinforcement LearningModel-based Reinforcement LearningMuJoCoPrediction

Methods 이 논문이 사용한 방법론

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Dirichlet-Based Monte Carlo Dropout for Uncertainty Estimation in Neural Networks

2026-05-22 · Rouaa Hoblos, Noura Dridi, Noureddine Zerhouni, Zeina Al Masry arxiv

Traditional neural networks provide deterministic predictions without inherent uncertainty estimates. While Bayesian Neural Networks (BNNs) offer a principled approach to uncertainty quantification, their computational c…

Computational Efficiency Bayesian Inference

Decision-Making in Reinforcement Learning

2019-06-01 · Arsh Javed Rehman, Pradeep Tomar

In this research work, probabilistic decision-making approaches are studied, e.g. Bayesian and Boltzmann strategies, along with various deterministic exploration strategies, e.g. greedy, epsilon-Greedy and random approac…

Decision Making Deep Reinforcement Learning OpenAI Gym reinforcement-learning+2

Concrete Dropout

2017-05-22 · NeurIPS 2017 12 · Yarin Gal, Jiri Hron, Alex Kendall

Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimates, a grid-search over the dropout probab…

Reinforcement Learning Reinforcement Learning (RL)

Uncertainty Tube Visualization of Particle Trajectories

2025-08-19 · Jixian Li, Timbwaoga Aime Judicael Ouermi, Mengjiao Han, Chris R. Johnson arxiv

Predicting particle trajectories with neural networks (NNs) has substantially enhanced many scientific and engineering domains. However, effectively quantifying and visualizing the inherent uncertainty in predictions rem…

Uncertainty Quantification In Surface Landmines and UXO Classification Using MC Dropout

2025-10-03 · Sagar Lekhak, Emmett J. Ientilucci, Dimah Dera, Susmita Ghosh arxiv

Detecting surface landmines and unexploded ordnances (UXOs) using deep learning has shown promise in humanitarian demining. However, deterministic neural networks can be vulnerable to noisy conditions and adversarial att…