Adaptive Experimental Design and Counterfactual Inference
Adaptive experimental design methods are increasingly being used in industry as a tool to boost testing throughput or reduce experimentation cost relative to traditional A/B/N testing methods. This paper shares lessons learned regarding the challenges and pitfalls of naively using adaptive experimentation systems in industrial settings where non-stationarity is prevalent, while also providing perspectives on the proper objectives and system specifications in these settings. We developed an adaptive experimental design framework for counterfactual inference based on these experiences, and tested it in a commercial environment.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualCounterfactual InferenceExperimental DesignSimilar Papers 제목 키워드 기반
Best of Three Worlds: Adaptive Experimentation for Digital Marketing in Practice
Adaptive experimental design (AED) methods are increasingly being used in industry as a tool to boost testing throughput or reduce experimentation cost relative to traditional A/B/N testing methods. However, the behavior…
counterfactualCounterfactual InferenceExperimental DesignMarketingGeometry Adaptive Counterfactual Distribution Learning with Diffusion-Guided Smoothing
We study counterfactual distribution learning for high-dimensional outcomes whose counterfactual law may concentrate near lower-dimensional structure. Standard isotropic smoothing treats all ambient directions equally, l…
Counterfactual inference for sequential experiments
We consider after-study statistical inference for sequentially designed experiments wherein multiple units are assigned treatments for multiple time points using treatment policies that adapt over time. Our goal is to pr…
counterfactualCounterfactual InferenceExperimental DesignMatrix Completion+1CounterBench: A Benchmark for Counterfactuals Reasoning in Large Language Models
Counterfactual reasoning is widely recognized as one of the most challenging and intricate aspects of causality in artificial intelligence. In this paper, we evaluate the performance of large language models (LLMs) in co…
Commonsense Causal ReasoningcounterfactualCounterfactual InferenceCounterfactual ReasoningMultiVerse: Causal Reasoning using Importance Sampling in Probabilistic Programming
We elaborate on using importance sampling for causal reasoning, in particular for counterfactual inference. We show how this can be implemented natively in probabilistic programming. By considering the structure of the c…
counterfactualCounterfactual InferenceProbabilistic Programming