Robust Sequential Experimental Design for A/B Testing
Experimental design has emerged as a powerful approach for improving the sample efficiency of A/B testing, yet existing designs rely critically on correctly specified models. We study robust sequential experimental design under model misspecification and develop a unified framework that covers both contextual bandit and dynamic settings. Theoretically, we prove that our design bounds the worst-case mean squared error of the estimated treatment effect. Empirically, we demonstrate the effectiveness of the proposed approach using synthetic and real-world datasets from a leading technology company.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Noisy Adaptive Group Testing using Bayesian Sequential Experimental Design
When the infection prevalence of a disease is low, Dorfman showed 80 years ago that testing groups of people can prove more efficient than testing people individually. Our goal in this paper is to propose new group testi…
DecoderExperimental DesignEfficient Adaptive Experimental Design for Average Treatment Effect Estimation
We study how to efficiently estimate average treatment effects (ATEs) using adaptive experiments. In adaptive experiments, experimenters sequentially assign treatments to experimental units while updating treatment assig…
Experimental DesignvalidA Bandit Approach to Sequential Experimental Design with False Discovery Control
We propose a new adaptive sampling approach to multiple testing which aims to maximize statistical power while ensuring anytime false discovery control. We consider $n$ distributions whose means are partitioned by whethe…
Drug DiscoveryExperimental DesignMulti-Armed BanditsRapid and Scalable Bayesian AB Testing
AB testing aids business operators with their decision making, and is considered the gold standard method for learning from data to improve digital user experiences. However, there is usually a gap between the requiremen…
Decision MakingFast Cross-Validation via Sequential Testing
With the increasing size of today's data sets, finding the right parameter configuration in model selection via cross-validation can be an extremely time-consuming task. In this paper we propose an improved cross-validat…
Model Selection