An Outcome Test of Discrimination for Ranked Lists
This paper extends Becker (1957)'s outcome test of discrimination to settings where a (human or algorithmic) decision-maker produces a ranked list of candidates. Ranked lists are particularly relevant in the context of online platforms that produce search results or feeds, and also arise when human decisionmakers express ordinal preferences over a list of candidates. We show that non-discrimination implies a system of moment inequalities, which intuitively impose that one cannot permute the position of a lower-ranked candidate from one group with a higher-ranked candidate from a second group and systematically improve the objective. Moreover, we show that that these moment inequalities are the only testable implications of non-discrimination when the auditor observes only outcomes and group membership by rank. We show how to statistically test the implied inequalities, and validate our approach in an application using data from LinkedIn.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
An information theoretic approach to quantify the stability of feature selection and ranking algorithms
Feature selection is a key step when dealing with high dimensional data. In particular, these techniques simplify the process of knowledge discovery from the data by selecting the most relevant features out of the noisy,…
feature selectionA Versatile Framework for Evaluating Ranked Lists in terms of Group Fairness and Relevance
We present a simple and versatile framework for evaluating ranked lists in terms of group fairness and relevance, where the groups (i.e., possible attribute values) can be either nominal or ordinal in nature. First, we d…
AttributeFairnessOn Discrimination Discovery and Removal in Ranked Data using Causal Graph
Predictive models learned from historical data are widely used to help companies and organizations make decisions. However, they may digitally unfairly treat unwanted groups, raising concerns about fairness and discrimin…
FairnessLanguage Fairness in Multilingual Information Retrieval
Multilingual information retrieval (MLIR) considers the problem of ranking documents in several languages for a query expressed in a language that may differ from any of those languages. Recent work has observed that app…
FairnessInformation RetrievalRetrievalOffline Evaluation of Ranked Lists using Parametric Estimation of Propensities
Search engines and recommendation systems attempt to continually improve the quality of the experience they afford to their users. Refining the ranker that produces the lists displayed in response to user requests is an …
Learning-To-RankRecommendation Systems