paper-with-me

Papers

On Strategyproof Conference Peer Review

2018-06-16 · Yichong Xu, Han Zhao, Xiaofei Shi, Jeremy Zhang, Nihar B. Shah

We consider peer review in a conference setting where there is typically an overlap between the set of reviewers and the set of authors. This overlap can incentivize strategic reviews to influence the final ranking of one's own papers. In this work, we address this problem through the lens of social choice, and present a theoretical framework for strategyproof and efficient peer review. We first present and analyze an algorithm for reviewer-assignment and aggregation that guarantees strategyproofness and a natural efficiency property called unanimity, when the authorship graph satisfies a simple property. Our algorithm is based on the so-called partitioning method, and can be thought as a generalization of this method to conference peer review settings. We then empirically show that the requisite property on the authorship graph is indeed satisfied in the submission data from the ICLR conference, and further demonstrate a simple trick to make the partitioning method more practically appealing for conference peer review. Finally, we complement our positive results with negative theoretical results where we prove that under various ways of strengthening the requirements, it is impossible for any algorithm to be strategyproof and efficient.

📄 PDF Abstract BibTeX arXiv:1806.06266

Code (1)

xycforgithub/StrategyProof_Conference_Review 공식 구현

Similar Papers 제목 키워드 기반

Strategyproofing Peer Assessment via Partitioning: The Price in Terms of Evaluators' Expertise

2022-01-25 · Komal Dhull, Steven Jecmen, Pravesh Kothari, Nihar B. Shah

Strategic behavior is a fundamental problem in a variety of real-world applications that require some form of peer assessment, such as peer grading of homeworks, grant proposal review, conference peer review of scientifi…

Automatic Analysis of Substantiation in Scientific Peer Reviews

2023-11-20 · Yanzhu Guo, Guokan Shang, Virgile Rennard, Michalis Vazirgiannis 외

With the increasing amount of problematic peer reviews in top AI conferences, the community is urgently in need of automatic quality control measures. In this paper, we restrict our attention to substantiation -- one pop…

Argument Mining

Strategyproof Peer Selection using Randomization, Partitioning, and Apportionment

2016-04-13 · Haris Aziz, Omer Lev, Nicholas Mattei, Jeffrey S. Rosenschein 외

Peer reviews, evaluations, and selections are a fundamental aspect of modern science. Funding bodies the world over employ experts to review and select the best proposals from those submitted for funding. The problem of …

Decision MakingMarketing

Paper Copilot: Tracking the Evolution of Peer Review in AI Conferences

2025-10-15 · Jing Yang, Qiyao Wei, Jiaxin Pei arxiv

The rapid growth of AI conferences is straining an already fragile peer-review system, leading to heavy reviewer workloads, expertise mismatches, inconsistent evaluation standards, superficial or templated reviews, and l…

What Factors Should Paper-Reviewer Assignments Rely On? Community Perspectives on Issues and Ideals in Conference Peer-Review

2022-05-02 · NAACL 2022 7 · Terne Sasha Thorn Jakobsen, Anna Rogers

Both scientific progress and individual researcher careers depend on the quality of peer review, which in turn depends on paper-reviewer matching. Surprisingly, this problem has been mostly approached as an automated rec…