Posterior Sampling for Large Scale Reinforcement Learning

2017-11-21 · Georgios Theocharous, Zheng Wen, Yasin Abbasi-Yadkori, Nikos Vlassis

We propose a practical non-episodic PSRL algorithm that unlike recent state-of-the-art PSRL algorithms uses a deterministic, model-independent episode switching schedule. Our algorithm termed deterministic schedule PSRL (DS-PSRL) is efficient in terms of time, sample, and space complexity. We prove a Bayesian regret bound under mild assumptions. Our result is more generally applicable to multiple parameters and continuous state action problems. We compare our algorithm with state-of-the-art PSRL algorithms on standard discrete and continuous problems from the literature. Finally, we show how the assumptions of our algorithm satisfy a sensible parametrization for a large class of problems in sequential recommendations.

📄 PDF Abstract BibTeX arXiv:1711.07979

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Provably Efficient Model-Free Posterior Sampling Method for Episodic Reinforcement Learning

2022-08-23 · NeurIPS 2021 12 · Christoph Dann, Mehryar Mohri, Tong Zhang, Julian Zimmert

Thompson Sampling is one of the most effective methods for contextual bandits and has been generalized to posterior sampling for certain MDP settings. However, existing posterior sampling methods for reinforcement learni…

Multi-Armed Bandits reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Posterior Sampling for Deep Reinforcement Learning

2023-04-30 · Remo Sasso, Michelangelo Conserva, Paulo Rauber

Despite remarkable successes, deep reinforcement learning algorithms remain sample inefficient: they require an enormous amount of trial and error to find good policies. Model-based algorithms promise sample efficiency b…

Computational Efficiency Deep Reinforcement Learning Model-based Reinforcement Learning reinforcement-learning+3

Efficient Model-Free Reinforcement Learning Using Gaussian Process

2018-12-11 · Ying Fan, Letian Chen, Yizhou Wang

Efficient Reinforcement Learning usually takes advantage of demonstration or good exploration strategy. By applying posterior sampling in model-free RL under the hypothesis of GP, we propose Gaussian Process Posterior Sa…

model reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-agent Reinforcement Learning Accelerated MCMC on Multiscale Inversion Problem

2020-11-17 · Eric Chung, Yalchin Efendiev, Wing Tat Leung, Sai-Mang Pun 외

In this work, we propose a multi-agent actor-critic reinforcement learning (RL) algorithm to accelerate the multi-level Monte Carlo Markov Chain (MCMC) sampling algorithms. The policies (actors) of the agents are used to…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Posterior Sampling-based Online Learning for Episodic POMDPs

2023-10-16 · Dengwang Tang, Dongze Ye, Rahul Jain, Ashutosh Nayyar 외

Learning in POMDPs is known to be significantly harder than in MDPs. In this paper, we consider the online learning problem for episodic POMDPs with unknown transition and observation models. We propose a Posterior Sampl…