Comparison Lift: Bandit-based Experimentation System for Online Advertising
Comparison Lift is an experimentation-as-a-service (EaaS) application for testing online advertising audiences and creatives at JD.com. Unlike many other EaaS tools that focus primarily on fixed sample A/B testing, Comparison Lift deploys a custom bandit-based experimentation algorithm. The advantages of the bandit-based approach are two-fold. First, it aligns the randomization induced in the test with the advertiser's goals from testing. Second, by adapting experimental design to information acquired during the test, it reduces substantially the cost of experimentation to the advertiser. Since launch in May 2019, Comparison Lift has been utilized in over 1,500 experiments. We estimate that utilization of the product has helped increase click-through rates of participating advertising campaigns by 46% on average. We estimate that the adaptive design in the product has generated 27% more clicks on average during testing compared to a fixed sample A/B design. Both suggest significant value generation and cost savings to advertisers from the product.
Code (0)
등록된 구현이 없습니다.
Tasks
Experimental DesignSimilar Papers 제목 키워드 기반
An Opportunistic Bandit Approach for User Interface Experimentation
Facing growing competition from online rivals, the retail industry is increasingly investing in their online shopping platforms to win the high-stake battle of customer' loyalty. User experience is playing an essential r…
Unifying Clustered and Non-stationary Bandits
Non-stationary bandits and online clustering of bandits lift the restrictive assumptions in contextual bandits and provide solutions to many important real-world scenarios. Though the essence in solving these two problem…
Change DetectionClusteringMulti-Armed BanditsOnline ClusteringMABWiser: A Parallelizable Contextual Multi-Armed Bandit Library for Python
Contextual multi-armed bandit algorithms serve as an effective technique to address online sequential decision-making problems. Despite their popularity, when it comes to off-the-shelf tools the library support remains l…
Decision MakingSequential Decision MakingBudget-Constrained Causal Bandits: Bridging Uplift Modeling and Sequential Decision-Making
Treatment allocation under budget constraints is a central challenge in digital advertising. The standard approach trains an offline uplift model on historical data, then solves a constrained optimization to allocate bud…
Parallel Experimentation and Competitive Interference on Online Advertising Platforms
This paper studies the measurement of advertising effects on online platforms when parallel experimentation occurs, that is, when multiple advertisers experiment concurrently. It provides a framework that makes precise h…
Decision MakingExperimental Design