Policy Evaluation with Latent Confounders via Optimal Balance
Evaluating novel contextual bandit policies using logged data is crucial in applications where exploration is costly, such as medicine. But it usually relies on the assumption of no unobserved confounders, which is bound to fail in practice. We study the question of policy evaluation when we instead have proxies for the latent confounders and develop an importance weighting method that avoids fitting a latent outcome regression model. We show that unlike the unconfounded case no single set of weights can give unbiased evaluation for all outcome models, yet we propose a new algorithm that can still provably guarantee consistency by instead minimizing an adversarial balance objective. We further develop tractable algorithms for optimizing this objective and demonstrate empirically the power of our method when confounders are latent.
Code (1)
Tasks
regressionSimilar Papers 제목 키워드 기반
Off-policy Evaluation in Infinite-Horizon Reinforcement Learning with Latent Confounders
Off-policy evaluation (OPE) in reinforcement learning is an important problem in settings where experimentation is limited, such as education and healthcare. But, in these very same settings, observed actions are often c…
Off-policy evaluationreinforcement-learningReinforcement Learning (RL)A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes
We consider off-policy evaluation (OPE) in Partially Observable Markov Decision Processes (POMDPs), where the evaluation policy depends only on observable variables and the behavior policy depends on unobservable latent …
Off-policy evaluationConfounder Balancing for Instrumental Variable Regression with Latent Variable
This paper studies the confounding effects from the unmeasured confounders and the imbalance of observed confounders in IV regression and aims at unbiased causal effect estimation. Recently, nonlinear IV estimators were …
regressionvalidModel-Free and Model-Based Policy Evaluation when Causality is Uncertain
When decision-makers can directly intervene, policy evaluation algorithms give valid causal estimates. In off-policy evaluation (OPE), there may exist unobserved variables that both impact the dynamics and are used by th…
modelOff-policy evaluationSensitivityvalidCausal Effect Inference with Deep Latent-Variable Models
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for policy makers. The most important aspect of…
Causal Inference