CPM-sensitive AUC for CTR prediction
The prediction of click-through rate (CTR) is crucial for industrial applications, such as online advertising. AUC is a commonly used evaluation indicator for CTR models. For advertising platforms, online performance is generally evaluated by CPM. However, in practice, AUC often improves in offline evaluation, but online CPM does not. As a result, a huge waste of precious online traffic and human costs has been caused. This is because there is a gap between offline AUC and online CPM. AUC can only reflect the order on CTR, but it does not reflect the order of CTR*Bid. Moreover, the bids of different advertisements are different, so the loss of income caused by different reverse-order pair is also different. For this reason, we propose the CPM-sensitive AUC (csAUC) to solve all these problems. We also give the csAUC calculation method based on dynamic programming. It can fully support the calculation of csAUC on large-scale data in real-world applications.
Code (0)
등록된 구현이 없습니다.
Tasks
Click-Through Rate PredictionPredictionSimilar Papers 제목 키워드 기반
Towards Assumption-free Bias Mitigation
Despite the impressive prediction ability, machine learning models show discrimination towards certain demographics and suffer from unfair prediction behaviors. To alleviate the discrimination, extensive studies focus on…
PredictionvalidAdversarial Fairness Network
Fairness is becoming a rising concern in machine learning. Recent research has discovered that state-of-the-art models are amplifying social bias by making biased predictions towards some population groups (characterized…
BIG-bench Machine LearningFairnessRisk-Sensitive Conformal Prediction for Catheter Placement Detection in Chest X-rays
This paper presents a novel approach to catheter and line position detection in chest X-rays, combining multi-task learning with risk-sensitive conformal prediction to address critical clinical requirements. Our model si…
Conformal PredictionMulti-Task LearningPredictionWhen Fairness Meets Privacy: Fair Classification with Semi-Private Sensitive Attributes
Machine learning models have demonstrated promising performance in many areas. However, the concerns that they can be biased against specific demographic groups hinder their adoption in high-stake applications. Thus, it …
ClassificationFairnessOf Dice and Games: A Theory of Generalized Boosting
Cost-sensitive loss functions are crucial in many real-world prediction problems, where different types of errors are penalized differently; for example, in medical diagnosis, a false negative prediction can lead to wors…
Binary ClassificationLearning TheoryMedical DiagnosisPAC learning+1