Bayesian Policy Gradients via Alpha Divergence Dropout Inference

2017-12-06 · Peter Henderson, Thang Doan, Riashat Islam, David Meger

Policy gradient methods have had great success in solving continuous control tasks, yet the stochastic nature of such problems makes deterministic value estimation difficult. We propose an approach which instead estimates a distribution by fitting the value function with a Bayesian Neural Network. We optimize an $\alpha$-divergence objective with Bayesian dropout approximation to learn and estimate this distribution. We show that using the Monte Carlo posterior mean of the Bayesian value function distribution, rather than a deterministic network, improves stability and performance of policy gradient methods in continuous control MuJoCo simulations.

📄 PDF Abstract BibTeX arXiv:1712.02037

Code (1)

Breakend/BayesianPolicyGradients 공식 구현 tf

Tasks

continuous-control Continuous ControlMuJoCoPolicy Gradient Methods

Methods 이 논문이 사용한 방법론

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Dropout Inference in Bayesian Neural Networks with Alpha-divergences

2017-03-08 · ICML 2017 8 · Yingzhen Li, Yarin Gal

To obtain uncertainty estimates with real-world Bayesian deep learning models, practical inference approximations are needed. Dropout variational inference (VI) for example has been used for machine vision and medical ap…

Variational Inference

Alpha-Divergences in Variational Dropout

2017-11-12 · Bogdan Mazoure, Riashat Islam

We investigate the use of alternative divergences to Kullback-Leibler (KL) in variational inference(VI), based on the Variational Dropout \cite{kingma2015}. Stochastic gradient variational Bayes (SGVB) \cite{aevb} is a g…

Variational Inference

Uncertainty Estimates for Efficient Neural Network-based Dialogue Policy Optimisation

2017-11-30 · Christopher Tegho, Paweł Budzianowski, Milica Gašić

In statistical dialogue management, the dialogue manager learns a policy that maps a belief state to an action for the system to perform. Efficient exploration is key to successful policy optimisation. Current deep reinf…

Deep Reinforcement Learning Dialogue Management Efficient Exploration Efficient Neural Network+2

Information Geometry and Classical Cramér-Rao Type Inequalities

2021-04-02 · Kumar Vijay Mishra, M. Ashok Kumar

We examine the role of information geometry in the context of classical Cram\'er-Rao (CR) type inequalities. In particular, we focus on Eguchi's theory of obtaining dualistic geometric structures from a divergence functi…

Vocal Bursts Type Prediction

Black-box $α$-divergence Minimization

2015-11-10 · José Miguel Hernández-Lobato, Yingzhen Li, Mark Rowland, Daniel Hernández-Lobato 외

Black-box alpha (BB-$\alpha$) is a new approximate inference method based on the minimization of $\alpha$-divergences. BB-$\alpha$ scales to large datasets because it can be implemented using stochastic gradient descent.…

General Classification regression