RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions
Mobile health leverages personalized and contextually tailored interventions optimized through bandit and reinforcement learning algorithms. In practice, however, challenges such as participant heterogeneity, nonstationarity, and nonlinear relationships hinder algorithm performance. We propose RoME, a Robust Mixed-Effects contextual bandit algorithm that simultaneously addresses these challenges via (1) modeling the differential reward with user- and time-specific random effects, (2) network cohesion penalties, and (3) debiased machine learning for flexible estimation of baseline rewards. We establish a high-probability regret bound that depends solely on the dimension of the differential-reward model, enabling us to achieve robust regret bounds even when the baseline reward is highly complex. We demonstrate the superior performance of the RoME algorithm in a simulation and two off-policy evaluation studies.
Code (1)
Tasks
Multi-Armed BanditsOff-policy evaluationThompson SamplingSimilar Papers 제목 키워드 기반
Contextual Bandit with Herding Effects: Algorithms and Recommendation Applications
Contextual bandits serve as a fundamental algorithmic framework for optimizing recommendation decisions online. Though extensive attention has been paid to tailoring contextual bandits for recommendation applications, th…
Multi-Armed BanditsThompson SamplingA Sleeping, Recovering Bandit Algorithm for Optimizing Recurring Notifications
Many online and mobile applications rely on daily emails and push notifications to increase and maintain user engagement. The multi-armed bandit approach provides a useful framework for optimizing the content of these no…
Multi-Armed BanditsConcave Statistical Utility Maximization Bandits via Influence-Function Gradients
We study stochastic multi-armed bandits in which the objective is a statistical functional of the long-run reward distribution, rather than expected reward alone. Under mild continuity assumptions, we show that the infin…
Multi-Armed BanditsMOCA-HESP: Meta High-dimensional Bayesian Optimization for Combinatorial and Mixed Spaces via Hyper-ellipsoid Partitioning
High-dimensional Bayesian Optimization (BO) has attracted significant attention in recent research. However, existing methods have mainly focused on optimizing in continuous domains, while combinatorial (ordinal and cate…
Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRL
Despite a series of recent successes in reinforcement learning (RL), many RL algorithms remain sensitive to hyperparameters. As such, there has recently been interest in the field of AutoRL, which seeks to automate desig…
Data AugmentationHyperparameter OptimizationReinforcement Learning (RL)