paper-with-me

홈 › Papers

Online Pricing with Offline Data: Phase Transition and Inverse Square Law

2019-10-19 · ICML 2020 1 · Jinzhi Bu, David Simchi-Levi, Yunzong Xu

This paper investigates the impact of pre-existing offline data on online learning, in the context of dynamic pricing. We study a single-product dynamic pricing problem over a selling horizon of $T$ periods. The demand in each period is determined by the price of the product according to a linear demand model with unknown parameters. We assume that before the start of the selling horizon, the seller already has some pre-existing offline data. The offline data set contains $n$ samples, each of which is an input-output pair consisting of a historical price and an associated demand observation. The seller wants to utilize both the pre-existing offline data and the sequential online data to minimize the regret of the online learning process. We characterize the joint effect of the size, location and dispersion of the offline data on the optimal regret of the online learning process. Specifically, the size, location and dispersion of the offline data are measured by the number of historical samples $n$, the distance between the average historical price and the optimal price $\delta$, and the standard deviation of the historical prices $\sigma$, respectively. We show that the optimal regret is $\widetilde \Theta\left(\sqrt{T}\wedge \frac{T}{(n\wedge T)\delta^2+n\sigma^2}\right)$, and design a learning algorithm based on the "optimism in the face of uncertainty" principle, whose regret is optimal up to a logarithmic factor. Our results reveal surprising transformations of the optimal regret rate with respect to the size of the offline data, which we refer to as phase transitions. In addition, our results demonstrate that the location and dispersion of the offline data also have an intrinsic effect on the optimal regret, and we quantify this effect via the inverse-square law.

📄 PDF Abstract BibTeX arXiv:1910.08693

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning

2024-06-12 · Mohammadreza Nakhaei, Aidan Scannell, Joni Pajarinen

Offline reinforcement learning (RL) allows learning sequential behavior from fixed datasets. Since offline datasets do not cover all possible situations, many methods collect additional data during online fine-tuning to …

D4RLMuJoCoReinforcement Learning (RL)

Efficient and Uncertainty-Aware Diffusion Framework for Offline-to-Online Reinforcement Learning

2026-05-29 · Ha Manh Bui, Metod Jazbec, Eric Nalisnick, Anqi Liu arxiv

Offline-to-Online Reinforcement Learning (O2O-RL) leverages an offline, pre-trained policy to minimize costly online interactions. Although data-efficient, O2O-RL is susceptible to shifts between offline and online distr…

Reinforcement Learning

How does online shopping affect offline price sensitivity?

2025-06-18 · Shirsho Biswas, Hema Yoganarasimhan, Haonan Zhang

The rapid rise of e-commerce has transformed consumer behavior, prompting questions about how online adoption influences offline shopping. We examine whether consumers who adopt online shopping with a retailer become mor…

counterfactualSensitivity

Contextual Online Pricing with (Biased) Offline Data

2025-07-03 · Yixuan Zhang, Ruihao Zhu, Qiaomin Xie arxiv

We study contextual online pricing with biased offline data. For the scalar price elasticity case, we identify the instance-dependent quantity $δ^2$ that measures how far the offline data lies from the (unknown) online o…

Offline-to-Online Multi-Agent Reinforcement Learning with Offline Value Function Memory and Sequential Exploration

2024-10-25 · Hai Zhong, Xun Wang, Zhuoran Li, Longbo Huang

Offline-to-Online Reinforcement Learning has emerged as a powerful paradigm, leveraging offline data for initialization and online fine-tuning to enhance both sample efficiency and performance. However, most existing res…

Efficient ExplorationMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+3