Strategyproof Peer Selection using Randomization, Partitioning, and Apportionment
Peer reviews, evaluations, and selections are a fundamental aspect of modern science. Funding bodies the world over employ experts to review and select the best proposals from those submitted for funding. The problem of peer selection, however, is much more general: a professional society may want to give a subset of its members awards based on the opinions of all members; an instructor for a Massive Open Online Course (MOOC) or an online course may want to crowdsource grading; or a marketing company may select ideas from group brainstorming sessions based on peer evaluation. We make three fundamental contributions to the study of peer selection, a specific type of group decision-making problem, studied in computer science, economics, and political science. First, we propose a novel mechanism that is strategyproof, i.e., agents cannot benefit by reporting insincere valuations. Second, we demonstrate the effectiveness of our mechanism by a comprehensive simulation-based comparison with a suite of mechanisms found in the literature. Finally, our mechanism employs a randomized rounding technique that is of independent interest, as it solves the apportionment problem that arises in various settings where discrete resources such as parliamentary representation slots need to be divided proportionally.
Code (1)
Tasks
Decision MakingMarketingSimilar Papers 제목 키워드 기반
Strategyproofing Peer Assessment via Partitioning: The Price in Terms of Evaluators' Expertise
Strategic behavior is a fundamental problem in a variety of real-world applications that require some form of peer assessment, such as peer grading of homeworks, grant proposal review, conference peer review of scientifi…
On Strategyproof Conference Peer Review
We consider peer review in a conference setting where there is typically an overlap between the set of reviewers and the set of authors. This overlap can incentivize strategic reviews to influence the final ranking of on…
Peer Selection with Noisy Assessments
In the peer selection problem a group of agents must select a subset of themselves as winners for, e.g., peer-reviewed grants or prizes. Here, we take a Condorcet view of this aggregation problem, i.e., that there is a g…
Non-Obvious Manipulability of the Rank-Minimizing Mechanism
In assignment problems, the rank distribution of assigned objects is often used to evaluate match quality. Rank-minimizing (RM) mechanisms directly optimize for average rank. While appealing, a drawback is RM mechanisms …
PeerNomination: Relaxing Exactness for Increased Accuracy in Peer Selection
In peer selection agents must choose a subset of themselves for an award or a prize. As agents are self-interested, we want to design algorithms that are impartial, so that an individual agent cannot affect their own cha…