IntelligentPooling: Practical Thompson Sampling for mHealth

2020-07-31 · Sabina Tomkins, Peng Liao, Predrag Klasnja, Susan Murphy

In mobile health (mHealth) smart devices deliver behavioral treatments repeatedly over time to a user with the goal of helping the user adopt and maintain healthy behaviors. Reinforcement learning appears ideal for learning how to optimally make these sequential treatment decisions. However, significant challenges must be overcome before reinforcement learning can be effectively deployed in a mobile healthcare setting. In this work we are concerned with the following challenges: 1) individuals who are in the same context can exhibit differential response to treatments 2) only a limited amount of data is available for learning on any one individual, and 3) non-stationary responses to treatment. To address these challenges we generalize Thompson-Sampling bandit algorithms to develop IntelligentPooling. IntelligentPooling learns personalized treatment policies thus addressing challenge one. To address the second challenge, IntelligentPooling updates each user's degree of personalization while making use of available data on other users to speed up learning. Lastly, IntelligentPooling allows responsivity to vary as a function of a user's time since beginning treatment, thus addressing challenge three. We show that IntelligentPooling achieves an average of 26% lower regret than state-of-the-art. We demonstrate the promise of this approach and its ability to learn from even a small group of users in a live clinical trial.

📄 PDF Abstract BibTeX arXiv:2008.01571

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Thompson Sampling

Rapidly Personalizing Mobile Health Treatment Policies with Limited Data

2020-02-23 · Sabina Tomkins, Peng Liao, Predrag Klasnja, Serena Yeung 외

In mobile health (mHealth), reinforcement learning algorithms that adapt to one's context without learning personalized policies might fail to distinguish between the needs of individuals. Yet the high amount of noise du…

Reinforcement Learning Reinforcement Learning (RL)

Thompson sampling for zero-inflated count outcomes with an application to the Drink Less mobile health study

2023-11-24 · Xueqing Liu, Nina Deliu, Tanujit Chakraborty, Lauren Bell 외

Mobile health (mHealth) interventions often aim to improve distal outcomes, such as clinical conditions, by optimizing proximal outcomes through just-in-time adaptive interventions. Contextual bandits provide a suitable …

Decision Making Multi-Armed Bandits Thompson Sampling

Bayesian Collaborative Bandits with Thompson Sampling for Improved Outreach in Maternal Health Program

2024-10-28 · Arpan Dasgupta, Gagan Jain, Arun Suggala, Karthikeyan Shanmugam 외

Mobile health (mHealth) programs face a critical challenge in optimizing the timing of automated health information calls to beneficiaries. This challenge has been formulated as a collaborative multi-armed bandit problem…

Matrix Completion Thompson Sampling

Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling

2018-02-26 · ICLR 2018 1 · Carlos Riquelme, George Tucker, Jasper Snoek

Recent advances in deep reinforcement learning have made significant strides in performance on applications such as Go and Atari games. However, developing practical methods to balance exploration and exploitation in com…

Decision Making Deep Reinforcement Learning Multi-Armed Bandits reinforcement-learning+4

Efficient and Adaptive Posterior Sampling Algorithms for Bandits

2024-05-02 · Bingshan Hu, Zhiming Huang, Tianyue H. Zhang, Mathias Lécuyer 외

We study Thompson Sampling-based algorithms for stochastic bandits with bounded rewards. As the existing problem-dependent regret bound for Thompson Sampling with Gaussian priors [Agrawal and Goyal, 2017] is vacuous when…

Thompson Sampling