Combining Offline Causal Inference and Online Bandit Learning for Data Driven Decision
A fundamental question for companies with large amount of logged data is: How to use such logged data together with incoming streaming data to make good decisions? Many companies currently make decisions via online A/B tests, but wrong decisions during testing hurt users' experiences and cause irreversible damage. A typical alternative is offline causal inference, which analyzes logged data alone to make decisions. However, these decisions are not adaptive to the new incoming data, and so a wrong decision will continuously hurt users' experiences. To overcome the aforementioned limitations, we propose a framework to unify offline causal inference algorithms (e.g., weighting, matching) and online learning algorithms (e.g., UCB, LinUCB). We propose novel algorithms and derive bounds on the decision accuracy via the notion of "regret". We derive the first upper regret bound for forest-based online bandit algorithms. Experiments on two real datasets show that our algorithms outperform other algorithms that use only logged data or online feedbacks, or algorithms that do not use the data properly.
Code (0)
등록된 구현이 없습니다.
Tasks
Causal InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach
This paper studies bandit problems where an agent has access to offline data that might be utilized to potentially improve the estimation of each arm's reward distribution. A major obstacle in this setting is the existen…
Selection biasBudget-Constrained Causal Bandits: Bridging Uplift Modeling and Sequential Decision-Making
Treatment allocation under budget constraints is a central challenge in digital advertising. The standard approach trains an offline uplift model on historical data, then solves a constrained optimization to allocate bud…
Causal Bandits: Learning Good Interventions via Causal Inference
We study the problem of using causal models to improve the rate at which good interventions can be learned online in a stochastic environment. Our formalism combines multi-arm bandits and causal inference to model a nove…
Causal InferenceContextual Multi-Armed Bandits for Causal Marketing
This work explores the idea of a causal contextual multi-armed bandit approach to automated marketing, where we estimate and optimize the causal (incremental) effects. Focusing on causal effect leads to better return on …
Causal InferencecounterfactualMarketingMulti-Armed Bandits+1Combining Online Learning and Offline Learning for Contextual Bandits with Deficient Support
We address policy learning with logged data in contextual bandits. Current offline-policy learning algorithms are mostly based on inverse propensity score (IPS) weighting requiring the logging policy to have \emph{full s…
Multi-Armed Bandits