PAC-Bayesian Policy Evaluation for Reinforcement Learning

2012-02-14 · Mahdi Milani Fard, Joelle Pineau, Csaba Szepesvari

Bayesian priors offer a compact yet general means of incorporating domain knowledge into many learning tasks. The correctness of the Bayesian analysis and inference, however, largely depends on accuracy and correctness of these priors. PAC-Bayesian methods overcome this problem by providing bounds that hold regardless of the correctness of the prior distribution. This paper introduces the first PAC-Bayesian bound for the batch reinforcement learning problem with function approximation. We show how this bound can be used to perform model-selection in a transfer learning scenario. Our empirical results confirm that PAC-Bayesian policy evaluation is able to leverage prior distributions when they are informative and, unlike standard Bayesian RL approaches, ignore them when they are misleading.

📄 PDF Abstract BibTeX arXiv:1202.3717

Code (0)

등록된 구현이 없습니다.

Tasks

Model Selectionreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Bayesian Residual Policy Optimization: Scalable Bayesian Reinforcement Learning with Clairvoyant Experts

2020-02-07 · Gilwoo Lee, Brian Hou, Sanjiban Choudhury, Siddhartha S. Srinivasa

Informed and robust decision making in the face of uncertainty is critical for robots that perform physical tasks alongside people. We formulate this as Bayesian Reinforcement Learning over latent Markov Decision Process…

Decision Making Policy Gradient Methods reinforcement-learning Reinforcement Learning+1

Nonparametric Bayesian Policy Priors for Reinforcement Learning

2010-12-01 · NeurIPS 2010 12 · Finale Doshi-Velez, David Wingate, Nicholas Roy, Joshua B. Tenenbaum

We consider reinforcement learning in partially observable domains where the agent can query an expert for demonstrations. Our nonparametric Bayesian approach combines model knowledge, inferred from expert information an…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Sequential Bayesian Optimal Experimental Design in Infinite Dimensions via Policy Gradient Reinforcement Learning

2026-01-09 · Kaichen Shen, Peng Chen arxiv

Sequential Bayesian optimal experimental design (SBOED) for PDE-governed inverse problems is computationally challenging, especially for infinite-dimensional random field parameters. High-fidelity approaches require repe…

Reinforcement Learning

Policy Learning for Malaria Control

2019-10-20 · Van Bach Nguyen, Belaid Mohamed Karim, Bao Long Vu, Jörg Schlötterer 외

Sequential decision making is a typical problem in reinforcement learning with plenty of algorithms to solve it. However, only a few of them can work effectively with a very small number of observations. In this report, …

Bayesian Optimization Decision Making Q-Learning reinforcement-learning+3

Bayesian Transfer Reinforcement Learning with Prior Knowledge Rules

2018-09-30 · Michalis K. Titsias, Sotirios Nikoloutsopoulos

We propose a probabilistic framework to directly insert prior knowledge in reinforcement learning (RL) algorithms by defining the behaviour policy as a Bayesian posterior distribution. Such a posterior combines task spec…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning+1