Meta-Thompson Sampling
Efficient exploration in bandits is a fundamental online learning problem. We propose a variant of Thompson sampling that learns to explore better as it interacts with bandit instances drawn from an unknown prior. The algorithm meta-learns the prior and thus we call it MetaTS. We propose several efficient implementations of MetaTS and analyze it in Gaussian bandits. Our analysis shows the benefit of meta-learning and is of a broader interest, because we derive a novel prior-dependent Bayes regret bound for Thompson sampling. Our theory is complemented by empirical evaluation, which shows that MetaTS quickly adapts to the unknown prior.
Code (0)
등록된 구현이 없습니다.
Tasks
Efficient ExplorationMeta-LearningMulti-Armed BanditsThompson SamplingSimilar Papers 제목 키워드 기반
Modified Meta-Thompson Sampling for Linear Bandits and Its Bayes Regret Analysis
Meta-learning is characterized by its ability to learn how to learn, enabling the adaptation of learning strategies across different tasks. Recent research introduced the Meta-Thompson Sampling (Meta-TS), which meta-lear…
Meta-LearningMulti-Armed BanditsThompson SamplingPrior-Aligned Meta-RL: Thompson Sampling with Learned Priors and Guarantees in Finite-Horizon MDPs
We study meta-reinforcement learning in finite-horizon MDPs where related tasks share similar structures in their optimal action-value functions. Specifically, we posit a linear representation $Q^*_h(s,a)=Φ_h(s,a)\,θ^{(k…
Reinforcement LearningMemory Sequence Length of Data Sampling Impacts the Adaptation of Meta-Reinforcement Learning Agents
Fast adaptation to new tasks is extremely important for embodied agents in the real world. Meta-reinforcement learning (meta-RL) has emerged as an effective method to enable fast adaptation in unknown environments. Compa…
continuous-controlContinuous ControlMeta Reinforcement LearningMuJoCo+1Meta Dynamic Pricing: Transfer Learning Across Experiments
We study the problem of learning shared structure \emph{across} a sequence of dynamic pricing experiments for related products. We consider a practical formulation where the unknown demand parameters for each product com…
Thompson SamplingTransfer LearningThompson sampling with the online bootstrap
Thompson sampling provides a solution to bandit problems in which new observations are allocated to arms with the posterior probability that an arm is optimal. While sometimes easy to implement and asymptotically optimal…
Thompson Sampling