Real-Time Bidding by Reinforcement Learning in Display Advertising

2017-01-10 · Han Cai, Kan Ren, Wei-Nan Zhang, Kleanthis Malialis, Jun Wang, Yong Yu, Defeng Guo

The majority of online display ads are served through real-time bidding (RTB) --- each ad display impression is auctioned off in real-time when it is just being generated from a user visit. To place an ad automatically and optimally, it is critical for advertisers to devise a learning algorithm to cleverly bid an ad impression in real-time. Most previous works consider the bid decision as a static optimization problem of either treating the value of each impression independently or setting a bid price to each segment of ad volume. However, the bidding for a given ad campaign would repeatedly happen during its life span before the budget runs out. As such, each bid is strategically correlated by the constrained budget and the overall effectiveness of the campaign (e.g., the rewards from generated clicks), which is only observed after the campaign has completed. Thus, it is of great interest to devise an optimal bidding strategy sequentially so that the campaign budget can be dynamically allocated across all the available impressions on the basis of both the immediate and future rewards. In this paper, we formulate the bid decision process as a reinforcement learning problem, where the state space is represented by the auction information and the campaign's real-time parameters, while an action is the bid price to set. By modeling the state transition via auction competition, we build a Markov Decision Process framework for learning the optimal bidding policy to optimize the advertising performance in the dynamic real-time bidding environment. Furthermore, the scalability problem from the large real-world auction volume and campaign budget is well handled by state value approximation using neural networks.

📄 PDF Abstract BibTeX arXiv:1701.02490

Code (1)

han-cai/rlb-dp 공식 구현 tf

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Real-time Bidding Strategy in Display Advertising: An Empirical Analysis

2022-11-30 · Mengjuan Liu, Zhengning Hu, Zhi Lai, Daiwei Zheng 외

Bidding strategies that help advertisers determine bidding prices are receiving increasing attention as more and more ad impressions are sold through real-time bidding systems. This paper first describes the problem and …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising

2018-02-27 · Junqi Jin, Chengru Song, Han Li, Kun Gai 외

Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only nee…

Clustering Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising

2018-02-23 · Di Wu, Xiujun Chen, Xun Yang, Hao Wang 외

Real-time bidding (RTB) is an important mechanism in online display advertising, where a proper bid for each page view plays an essential role for good marketing results. Budget constrained bidding is a typical scenario …

Marketing reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep Reinforcement Learning for Sponsored Search Real-time Bidding

2018-03-01 · Jun Zhao, Guang Qiu, Ziyu Guan, Wei Zhao 외

Bidding optimization is one of the most critical problems in online advertising. Sponsored search (SS) auction, due to the randomness of user query behavior and platform nature, usually adopts keyword-level bidding strat…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Multi-Agent Reinforcement Learning Method for Impression Allocation in Online Display Advertising

2018-09-10 · Di Wu, Cheng Chen, Xun Yang, Xiujun Chen 외

In online display advertising, guaranteed contracts and real-time bidding (RTB) are two major ways to sell impressions for a publisher. Despite the increasing popularity of RTB, there is still half of online display adve…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)