paper-with-me

Papers

Rethinking Position Bias Modeling with Knowledge Distillation for CTR Prediction

2022-04-01 · Congcong Liu, Yuejiang Li, Jian Zhu, Xiwei Zhao, Changping Peng, Zhangang Lin, Jingping Shao

Click-through rate (CTR) Prediction is of great importance in real-world online ads systems. One challenge for the CTR prediction task is to capture the real interest of users from their clicked items, which is inherently biased by presented positions of items, i.e., more front positions tend to obtain higher CTR values. A popular line of existing works focuses on explicitly estimating position bias by result randomization which is expensive and inefficient, or by inverse propensity weighting (IPW) which relies heavily on the quality of the propensity estimation. Another common solution is modeling position as features during offline training and simply adopting fixed value or dropout tricks when serving. However, training-inference inconsistency can lead to sub-optimal performance. Furthermore, post-click information such as position values is informative while less exploited in CTR prediction. This work proposes a simple yet efficient knowledge distillation framework to alleviate the impact of position bias and leverage position information to improve CTR prediction. We demonstrate the performance of our proposed method on a real-world production dataset and online A/B tests, achieving significant improvements over competing baseline models. The proposed method has been deployed in the real world online ads systems, serving main traffic on one of the world's largest e-commercial platforms.

📄 PDF Abstract BibTeX arXiv:2204.00270

Code (0)

등록된 구현이 없습니다.

Tasks

Click-Through Rate PredictionKnowledge DistillationPositionPrediction

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Similar Papers 제목 키워드 기반

Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective

2021-02-01 · Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou 외

Knowledge distillation is an effective approach to leverage a well-trained network or an ensemble of them, named as the teacher, to guide the training of a student network. The outputs from the teacher network are used a…

Knowledge Distillation

Rethinking Soft Labels for Knowledge Distillation: A Bias–Variance Tradeoff Perspective

2021-01-01 · ICLR 2021 1 · Helong Zhou, Liangchen Song, Jiajie Chen, Ye Zhou 외

Knowledge distillation is an effective approach to leverage a well-trained network or an ensemble of them, named as the teacher, to guide the training of a student network. The outputs from the teacher network are used …

Knowledge Distillation

Rethinking Selective Knowledge Distillation

2026-02-01 · Almog Tavor, Itay Ebenspanger, Neil Cnaan, Mor Geva arxiv

Growing efforts to improve knowledge distillation (KD) in large language models (LLMs) replace dense teacher supervision with selective distillation, which uses a subset of token positions, vocabulary classes, or trainin…

Knowledge Distillation

Rethinking Intermediate Layers design in Knowledge Distillation for Kidney and Liver Tumor Segmentation

2023-11-28 · Vandan Gorade, Sparsh Mittal, Debesh Jha, Ulas Bagci

Knowledge distillation (KD) has demonstrated remarkable success across various domains, but its application to medical imaging tasks, such as kidney and liver tumor segmentation, has encountered challenges. Many existing…

DiagnosticKnowledge DistillationTumor Segmentation

UKD: Debiasing Conversion Rate Estimation via Uncertainty-regularized Knowledge Distillation

2022-01-20 · Zixuan Xu, Penghui Wei, Weimin Zhang, Shaoguo Liu 외

In online advertising, conventional post-click conversion rate (CVR) estimation models are trained using clicked samples. However, during online serving the models need to estimate for all impression ads, leading to the …

Knowledge DistillationSelection bias