Thompson Sampling for Parameterized Markov Decision Processes with Uninformative Actions

2023-05-13 · Michael Gimelfarb, Michael Jong Kim

We study parameterized MDPs (PMDPs) in which the key parameters of interest are unknown and must be learned using Bayesian inference. One key defining feature of such models is the presence of "uninformative" actions that provide no information about the unknown parameters. We contribute a set of assumptions for PMDPs under which Thompson sampling guarantees an asymptotically optimal expected regret bound of $O(T^{-1})$, which are easily verified for many classes of problems such as queuing, inventory control, and dynamic pricing.

📄 PDF Abstract BibTeX arXiv:2305.07844

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian InferenceThompson Sampling

Thompson Sampling for Learning Parameterized Markov Decision Processes

2014-06-29 · Aditya Gopalan, Shie Mannor

We consider reinforcement learning in parameterized Markov Decision Processes (MDPs), where the parameterization may induce correlation across transition probabilities or rewards. Consequently, observing a particular sta…

Form reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

On Online Learning in Kernelized Markov Decision Processes

2019-11-04 · Sayak Ray Chowdhury, Aditya Gopalan

We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidence Bound (UCB) as well as Posterior or Th…

Thompson Sampling

No-Regret Thompson Sampling for Finite-Horizon Markov Decision Processes with Gaussian Processes

2025-10-23 · Jasmine Bayrooti, Sattar Vakili, Amanda Prorok, Carl Henrik Ek arxiv

Thompson sampling (TS) is a powerful and widely used strategy for sequential decision-making, with applications ranging from Bayesian optimization to reinforcement learning (RL). Despite its success, the theoretical foun…

Reinforcement Learning Gaussian Processes

A Tutorial on Thompson Sampling

2017-07-07 · Daniel Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband 외

Thompson sampling is an algorithm for online decision problems where actions are taken sequentially in a manner that must balance between exploiting what is known to maximize immediate performance and investing to accumu…

Active Learning Product Recommendation Reinforcement Learning Thompson Sampling

Thompson Sampling for Infinite-Horizon Discounted Decision Processes

2024-05-14 · Daniel Adelman, Cagla Keceli, Alba V. Olivares-Nadal

We model a Markov decision process, parametrized by an unknown parameter, and study the asymptotic behavior of a sampling-based algorithm, called Thompson sampling. The standard definition of regret is not always suitabl…