Posterior Probability Matters: Doubly-Adaptive Calibration for Neural Predictions in Online Advertising
Predicting user response probabilities is vital for ad ranking and bidding. We hope that predictive models can produce accurate probabilistic predictions that reflect true likelihoods. Calibration techniques aim to post-process model predictions to posterior probabilities. Field-level calibration -- which performs calibration w.r.t. to a specific field value -- is fine-grained and more practical. In this paper we propose a doubly-adaptive approach AdaCalib. It learns an isotonic function family to calibrate model predictions with the guidance of posterior statistics, and field-adaptive mechanisms are designed to ensure that the posterior is appropriate for the field value to be calibrated. Experiments verify that AdaCalib achieves significant improvement on calibration performance. It has been deployed online and beats previous approach.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Consistency Analysis for the Doubly Stochastic Dirichlet Process
This technical report proves components consistency for the Doubly Stochastic Dirichlet Process with exponential convergence of posterior probability. We also present the fundamental properties for DSDP as well as infere…
Calibrating Where It Matters: Constrained Temperature Scaling
We consider calibration of convolutional classifiers for diagnostic decision making. Clinical decision makers can use calibrated classifiers to minimise expected costs given their own cost function. Such functions are us…
Decision MakingDiagnosticA Multi-armed Bandit MCMC, with applications in sampling from doubly intractable posterior
Markov chain Monte Carlo (MCMC) algorithms are widely used to sample from complicated distributions, especially to sample from the posterior distribution in Bayesian inference. However, MCMC is not directly applicable wh…
Bayesian InferencePosterior Calibrated Training on Sentence Classification Tasks
Most classification models work by first predicting a posterior probability distribution over all classes and then selecting that class with the largest estimated probability. In many settings however, the quality of pos…
ClassificationGeneral ClassificationSentenceSentence ClassificationDesign-marginal calibration of Gaussian process predictive distributions: Bayesian and conformal approaches
We study the calibration of Gaussian process (GP) predictive distributions in the interpolation setting from a design-marginal perspective. Conditioning on the data and averaging over a design measure μ, we formalize μ-c…