Bootstrapped Thompson Sampling and Deep Exploration

2015-07-01 · Ian Osband, Benjamin Van Roy

This technical note presents a new approach to carrying out the kind of exploration achieved by Thompson sampling, but without explicitly maintaining or sampling from posterior distributions. The approach is based on a bootstrap technique that uses a combination of observed and artificially generated data. The latter serves to induce a prior distribution which, as we will demonstrate, is critical to effective exploration. We explain how the approach can be applied to multi-armed bandit and reinforcement learning problems and how it relates to Thompson sampling. The approach is particularly well-suited for contexts in which exploration is coupled with deep learning, since in these settings, maintaining or generating samples from a posterior distribution becomes computationally infeasible.

📄 PDF Abstract BibTeX arXiv:1507.00300

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Thompson Sampling

Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics

2019-03-11 · Denis Steckelmacher, Hélène Plisnier, Diederik M. Roijers, Ann Nowé

Value-based reinforcement-learning algorithms provide state-of-the-art results in model-free discrete-action settings, and tend to outperform actor-critic algorithms. We argue that actor-critic algorithms are limited by …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Thompson Sampling

Thompson Sampling via Local Uncertainty

2019-10-30 · ICML 2020 1 · Zhendong Wang, Mingyuan Zhou

Thompson sampling is an efficient algorithm for sequential decision making, which exploits the posterior uncertainty to address the exploration-exploitation dilemma. There has been significant recent interest in integrat…

Decision Making Multi-Armed Bandits Sequential Decision Making Thompson Sampling+1

Thompson Sampling and Approximate Inference

2019-12-01 · NeurIPS 2019 12 · My Phan, Yasin Abbasi Yadkori, Justin Domke

We study the effects of approximate inference on the performance of Thompson sampling in the $k$-armed bandit problems. Thompson sampling is a successful algorithm for online decision-making but requires posterior infere…

Decision Making Thompson Sampling

Thompson Sampling with Approximate Inference

2019-08-14 · NeurIPS 2019 12 · My Phan, Yasin Abbasi-Yadkori, Justin Domke

Decision Making Thompson Sampling

An Empirical Evaluation of Thompson Sampling

2011-12-01 · NeurIPS 2011 12 · Olivier Chapelle, Lihong Li

Thompson sampling is one of oldest heuristic to address the exploration / exploitation trade-off, but it is surprisingly not very popular in the literature. We present here some empirical results using Thompson sampling …

Multi-Armed Bandits Thompson Sampling