Discovery of Bias and Strategic Behavior in Crowdsourced Performance Assessment
With the industry trend of shifting from a traditional hierarchical approach to flatter management structure, crowdsourced performance assessment gained mainstream popularity. One fundamental challenge of crowdsourced performance assessment is the risks that personal interest can introduce distortions of facts, especially when the system is used to determine merit pay or promotion. In this paper, we developed a method to identify bias and strategic behavior in crowdsourced performance assessment, using a rich dataset collected from a professional service firm in China. We find a pattern of "discriminatory generosity" on the part of peer evaluation, where raters downgrade their peer coworkers who have passed objective promotion requirements while overrating their peer coworkers who have not yet passed. This introduces two types of biases: the first aimed against more competent competitors, and the other favoring less eligible peers which can serve as a mask of the first bias. This paper also aims to bring angles of fairness-aware data mining to talent and management computing. Historical decision records, such as performance ratings, often contain subjective judgment which is prone to bias and strategic behavior. For practitioners of predictive talent analytics, it is important to investigate potential bias and strategic behavior underlying historical decision records.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessManagementSimilar Papers 제목 키워드 기반
Truthful Incentive Mechanism for Federated Learning with Crowdsourced Data Labeling
Federated learning (FL) has emerged as a promising paradigm that trains machine learning (ML) models on clients' devices in a distributed manner without the need of transmitting clients' data to the FL server. In many ap…
Federated LearningDiscovering Differences in Strategic Behavior Between Humans and LLMs
As Large Language Models (LLMs) are increasingly deployed in social and strategic scenarios, it becomes critical to understand where and why their behavior diverges from that of humans. While behavioral game theory (BGT)…
The Double-Edged Sword of Behavioral Responses in Strategic Classification: Theory and User Studies
When humans are subject to an algorithmic decision system, they can strategically adjust their behavior accordingly (``game'' the system). While a growing line of literature on strategic classification has used game-theo…
False Discovery Rate Control and Statistical Quality Assessment of Annotators in Crowdsourced Ranking
With the rapid growth of crowdsourcing platforms it has become easy and relatively inexpensive to collect a dataset labeled by multiple annotators in a short time. However due to the lack of control over the quality of t…
PositionSociologyBeyond Rational Illusion: Behaviorally Realistic Strategic Classification
Strategic classification(SC) studies the interaction between decision models and agents who strategically manipulate their features for favorable outcomes. Existing SC frameworks typically rely on the idealized assumptio…