Causal Estimation of Position Bias in Recommender Systems Using Marketplace Instruments
Information retrieval systems, such as online marketplaces, news feeds, and search engines, are ubiquitous in today's digital society. They facilitate information discovery by ranking retrieved items on predicted relevance, i.e. likelihood of interaction (click, share) between users and items. Typically modeled using past interactions, such rankings have a major drawback: interaction depends on the attention items receive. A highly-relevant item placed outside a user's attention could receive little interaction. This discrepancy between observed interaction and true relevance is termed the position bias. Position bias degrades relevance estimation and when it compounds over time, it can silo users into false relevant items, causing marketplace inefficiencies. Position bias may be identified with randomized experiments, but such an approach can be prohibitive in cost and feasibility. Past research has also suggested propensity score methods, which do not adequately address unobserved confounding; and regression discontinuity designs, which have poor external validity. In this work, we address these concerns by leveraging the abundance of A/B tests in ranking evaluations as instrumental variables. Historical A/B tests allow us to access exogenous variation in rankings without manually introducing them, harming user experience and platform revenue. We demonstrate our methodology in two distinct applications at LinkedIn - feed ads and the People-You-May-Know (PYMK) recommender. The marketplaces comprise users and campaigns on the ads side, and invite senders and recipients on PYMK. By leveraging prior experimentation, we obtain quasi-experimental variation in item rankings that is orthogonal to user relevance. Our method provides robust position effect estimates that handle unobserved confounding well, greater generalizability, and easily extends to other information retrieval systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalPositionRecommendation SystemsRetrievalSimilar Papers 제목 키워드 기반
Cross-Dataset Propensity Estimation for Debiasing Recommender Systems
Datasets for training recommender systems are often subject to distribution shift induced by users' and recommenders' selection biases. In this paper, we study the impact of selection bias on datasets with different quan…
Causal InferenceQuantizationRecommendation SystemsSelection biasDeep Deconfounded Content-based Tag Recommendation for UGC with Causal Intervention
Traditional content-based tag recommender systems directly learn the association between user-generated content (UGC) and tags based on collected UGC-tag pairs. However, since a UGC uploader simultaneously creates the UG…
Recommendation SystemsTAGOn the Opportunity of Causal Learning in Recommendation Systems: Foundation, Estimation, Prediction and Challenges
Recently, recommender system (RS) based on causal inference has gained much attention in the industrial community, as well as the states of the art performance in many prediction and debiasing tasks. Nevertheless, a unif…
Causal InferencePredictionRecommendation SystemsDeep Causal Reasoning for Recommendations
Traditional recommender systems aim to estimate a user's rating to an item based on observed ratings from the population. As with all observational studies, hidden confounders, which are factors that affect both item exp…
Recommendation SystemsVariational InferenceCausal Inference for Recommendation: Foundations, Methods and Applications
Recommender systems are important and powerful tools for various personalized services. Traditionally, these systems use data mining and machine learning techniques to make recommendations based on correlations found in …
Causal InferenceFairnessRecommendation Systems