paper-with-me

홈 › Papers

Dual Instrumental Method for Confounded Kernelized Bandits

2022-09-07 · Xueping Gong, Jiheng Zhang

The contextual bandit problem is a theoretically justified framework with wide applications in various fields. While the previous study on this problem usually requires independence between noise and contexts, our work considers a more sensible setting where the noise becomes a latent confounder that affects both contexts and rewards. Such a confounded setting is more realistic and could expand to a broader range of applications. However, the unresolved confounder will cause a bias in reward function estimation and thus lead to a large regret. To deal with the challenges brought by the confounder, we apply the dual instrumental variable regression, which can correctly identify the true reward function. We prove the convergence rate of this method is near-optimal in two types of widely used reproducing kernel Hilbert spaces. Therefore, we can design computationally efficient and regret-optimal algorithms based on the theoretical guarantees for confounded bandit problems. The numerical results illustrate the efficacy of our proposed algorithms in the confounded bandit setting.

📄 PDF Abstract BibTeX arXiv:2209.03224

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Bias-Robust Bayesian Optimization via Dueling Bandits

2021-05-25 · Johannes Kirschner, Andreas Krause

We consider Bayesian optimization in settings where observations can be adversarially biased, for example by an uncontrolled hidden confounder. Our first contribution is a reduction of the confounded setting to the dueli…

Bayesian Optimization

Statistical Estimation of Confounded Linear MDPs: An Instrumental Variable Approach

2022-09-12 · Miao Lu, Wenhao Yang, Liangyu Zhang, Zhihua Zhang

In an Markov decision process (MDP), unobservable confounders may exist and have impacts on the data generating process, so that the classic off-policy evaluation (OPE) estimators may fail to identify the true value func…

Off-policy evaluation

Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning

2021-02-19 · Luofeng Liao, Zuyue Fu, Zhuoran Yang, Yixin Wang 외

In offline reinforcement learning (RL) an optimal policy is learned solely from a priori collected observational data. However, in observational data, actions are often confounded by unobserved variables. Instrumental va…

Offline RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1

Open Problem: Tight Bounds for Kernelized Multi-Armed Bandits with Bernoulli Rewards

2024-07-08 · Marco Mussi, Simone Drago, Alberto Maria Metelli

We consider Kernelized Bandits (KBs) to optimize a function $f : \mathcal{X} \rightarrow [0,1]$ belonging to the Reproducing Kernel Hilbert Space (RKHS) $\mathcal{H}_k$. Mainstream works on kernelized bandits focus on a …

Multi-Armed Bandits

Bandits with Partially Observable Confounded Data

2020-06-11 · Guy Tennenholtz, Uri Shalit, Shie Mannor, Yonathan Efroni

We study linear contextual bandits with access to a large, confounded, offline dataset that was sampled from some fixed policy. We show that this problem is closely related to a variant of the bandit problem with side in…

Multi-Armed Bandits