Bid Optimization using Maximum Entropy Reinforcement Learning

2021-10-11 · Mengjuan Liu, Jinyu Liu, Zhengning Hu, Yuchen Ge, Xuyun Nie

Real-time bidding (RTB) has become a critical way of online advertising. In RTB, an advertiser can participate in bidding ad impressions to display its advertisements. The advertiser determines every impression's bidding price according to its bidding strategy. Therefore, a good bidding strategy can help advertisers improve cost efficiency. This paper focuses on optimizing a single advertiser's bidding strategy using reinforcement learning (RL) in RTB. Unfortunately, it is challenging to optimize the bidding strategy through RL at the granularity of impression due to the highly dynamic nature of the RTB environment. In this paper, we first utilize a widely accepted linear bidding function to compute every impression's base price and optimize it by a mutable adjustment factor derived from the RTB auction environment, to avoid optimizing every impression's bidding price directly. Specifically, we use the maximum entropy RL algorithm (Soft Actor-Critic) to optimize the adjustment factor generation policy at the impression-grained level. Finally, the empirical study on a public dataset demonstrates that the proposed bidding strategy has superior performance compared with the baselines.

📄 PDF Abstract BibTeX arXiv:2110.05032

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Failure Modes of Maximum Entropy RLHF

2025-09-24 · Ömer Veysel Çağatan, Barış Akgün arxiv

In this paper, we show that Simple Preference Optimization (SimPO) can be derived as Maximum Entropy Reinforcement Learning, providing a theoretical foundation for this reference-free method. Motivated by SimPO's strong …

Reinforcement Learning

Generalized Maximum Entropy Reinforcement Learning via Reward Shaping

2021-09-29 · Feng Tao, Yongcan Cao

Entropy regularization is a commonly used technique in reinforcement learning to improve exploration and cultivate a better pre-trained policy for later adaptation. Recent studies further show that the use of entropy reg…

MuJoCo reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Maximum Causal Entropy Inverse Reinforcement Learning for Mean-Field Games

2024-01-12 · Berkay Anahtarci, Can Deha Kariksiz, Naci Saldi

In this paper, we introduce the maximum casual entropy Inverse Reinforcement Learning (IRL) problem for discrete-time mean-field games (MFGs) under an infinite-horizon discounted-reward optimality criterion. The state sp…

reinforcement-learning Reinforcement Learning

ELEMENT: Episodic and Lifelong Exploration via Maximum Entropy

2024-12-05 · Hongming Li, Shujian Yu, Bin Liu, Jose C. Principe

This paper proposes \emph{Episodic and Lifelong Exploration via Maximum ENTropy} (ELEMENT), a novel, multiscale, intrinsically motivated reinforcement learning (RL) framework that is able to explore environments without …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Average-Reward Maximum Entropy Reinforcement Learning for Underactuated Double Pendulum Tasks

2024-09-13 · Jean Seong Bjorn Choe, Bumkyu Choi, Jong-Kook Kim

This report presents a solution for the swing-up and stabilisation tasks of the acrobot and the pendubot, developed for the AI Olympics competition at IROS 2024. Our approach employs the Average-Reward Entropy Advantage …

Acrobot Reinforcement Learning (RL)