ROI-Constrained Bidding via Curriculum-Guided Bayesian Reinforcement Learning

2022-06-10 · Haozhe Wang, Chao Du, Panyan Fang, Shuo Yuan, Xuming He, Liang Wang, Bo Zheng

Real-Time Bidding (RTB) is an important mechanism in modern online advertising systems. Advertisers employ bidding strategies in RTB to optimize their advertising effects subject to various financial requirements, especially the return-on-investment (ROI) constraint. ROIs change non-monotonically during the sequential bidding process, and often induce a see-saw effect between constraint satisfaction and objective optimization. While some existing approaches show promising results in static or mildly changing ad markets, they fail to generalize to highly dynamic ad markets with ROI constraints, due to their inability to adaptively balance constraints and objectives amidst non-stationarity and partial observability. In this work, we specialize in ROI-Constrained Bidding in non-stationary markets. Based on a Partially Observable Constrained Markov Decision Process, our method exploits an indicator-augmented reward function free of extra trade-off parameters and develops a Curriculum-Guided Bayesian Reinforcement Learning (CBRL) framework to adaptively control the constraint-objective trade-off in non-stationary ad markets. Extensive experiments on a large-scale industrial dataset with two problem settings reveal that CBRL generalizes well in both in-distribution and out-of-distribution data regimes, and enjoys superior learning efficiency and stability.

📄 PDF Abstract BibTeX arXiv:2206.05240

Code (1)

HaozheJasper/CBRL_KDD22 공식 구현 pytorch

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement Learning

2023-12-29 · Hao Wang, Bo Tang, Chi Harold Liu, Shangqin Mao 외

Online display advertising platforms service numerous advertisers by providing real-time bidding (RTB) for the scale of billions of ad requests every day. The bidding strategy handles ad requests cross multiple channels …

Data Augmentation Deep Reinforcement Learning

Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising

2018-02-23 · Di Wu, Xiujun Chen, Xun Yang, Hao Wang 외

Real-time bidding (RTB) is an important mechanism in online display advertising, where a proper bid for each page view plays an essential role for good marketing results. Budget constrained bidding is a typical scenario …

Marketing reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

EBaReT: Expert-guided Bag Reward Transformer for Auto Bidding

2025-07-22 · Kaiyuan Li, Pengyu Wang, Yunshan Peng, Pengjia Yuan 외 arxiv

Reinforcement learning has been widely applied in automated bidding. Traditional approaches model bidding as a Markov Decision Process (MDP). Recently, some studies have explored using generative reinforcement learning m…

Reinforcement Learning

Adversarial Constrained Bidding via Minimax Regret Optimization with Causality-Aware Reinforcement Learning

2023-06-12 · Haozhe Wang, Chao Du, Panyan Fang, Li He 외

The proliferation of the Internet has led to the emergence of online advertising, driven by the mechanics of online auctions. In these repeated auctions, software agents participate on behalf of aggregated advertisers to…

Meta-Learning reinforcement-learning

Nash Equilibrium Constrained Auto-bidding With Bi-level Reinforcement Learning

2025-03-13 · Zhiyu Mou, Miao Xu, Rongquan Bai, Zhuoran Yang 외

Many online advertising platforms provide advertisers with auto-bidding services to enhance their advertising performance. However, most existing auto-bidding algorithms fail to accurately capture the auto-bidding proble…