Variational Inference MPC for Bayesian Model-based Reinforcement Learning

2019-07-08 · Masashi Okada, Tadahiro Taniguchi

In recent studies on model-based reinforcement learning (MBRL), incorporating uncertainty in forward dynamics is a state-of-the-art strategy to enhance learning performance, making MBRLs competitive to cutting-edge model free methods, especially in simulated robotics tasks. Probabilistic ensembles with trajectory sampling (PETS) is a leading type of MBRL, which employs Bayesian inference to dynamics modeling and model predictive control (MPC) with stochastic optimization via the cross entropy method (CEM). In this paper, we propose a novel extension to the uncertainty-aware MBRL. Our main contributions are twofold: Firstly, we introduce a variational inference MPC, which reformulates various stochastic methods, including CEM, in a Bayesian fashion. Secondly, we propose a novel instance of the framework, called probabilistic action ensembles with trajectory sampling (PaETS). As a result, our Bayesian MBRL can involve multimodal uncertainties both in dynamics and optimal trajectories. In comparison to PETS, our method consistently improves asymptotic performance on several challenging locomotion tasks.

📄 PDF Abstract BibTeX arXiv:1907.04202

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian InferenceModel-based Reinforcement LearningModel Predictive Controlreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Stochastic OptimizationVariational Inference

Variational Bayes: A report on approaches and applications

2019-05-26 · Manikanta Srikar Yellapragada, Chandra Prakash Konkimalla

Deep neural networks have achieved impressive results on a wide variety of tasks. However, quantifying uncertainty in the network's output is a challenging task. Bayesian models offer a mathematical framework to reason a…

Bayesian Inference Continual Learning reinforcement-learning Reinforcement Learning+2

Variational Inference for Policy Gradient

2018-02-21 · Tianbing Xu

Inspired by the seminal work on Stein Variational Inference and Stein Variational Policy Gradient, we derived a method to generate samples from the posterior variational parameter distribution by \textit{explicitly} mini…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Variational Inference

Bayesian Meta-Reinforcement Learning with Laplace Variational Recurrent Networks

2025-05-24 · Joery A. de Vries, Jinke He, Mathijs M. de Weerdt, Matthijs T. J. Spaan

Meta-reinforcement learning trains a single reinforcement learning agent on a distribution of tasks to quickly generalize to new tasks outside of the training set at test time. From a Bayesian perspective, one can interp…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning Variational Inference

Meta-RL with Bayesian Linear Task Models

2025-12-24 · Jingyang You, Hanna Kurniawati arxiv

Deep Bayesian reinforcement learning adapts to unseen tasks by inferring latent transition and reward models, but existing methods typically rely on variational posteriors and evidence lower bounds, introducing approxima…

Reinforcement Learning Bayesian Inference

Scalable Bayesian Inverse Reinforcement Learning

2021-02-12 · Alex J. Chan, Mihaela van der Schaar

Bayesian inference over the reward presents an ideal solution to the ill-posed nature of the inverse reinforcement learning problem. Unfortunately current methods generally do not scale well beyond the small tabular sett…

Bayesian Inference Imitation Learning reinforcement-learning Reinforcement Learning+1