paper-with-me

Papers

The AI Imperative: Scaling High-Quality Peer Review in Machine Learning

2025-06-09 · Qiyao Wei, Samuel Holt, Jing Yang, Markus Wulfmeier, Mihaela van der Schaar

Peer review, the bedrock of scientific advancement in machine learning (ML), is strained by a crisis of scale. Exponential growth in manuscript submissions to premier ML venues such as NeurIPS, ICML, and ICLR is outpacing the finite capacity of qualified reviewers, leading to concerns about review quality, consistency, and reviewer fatigue. This position paper argues that AI-assisted peer review must become an urgent research and infrastructure priority. We advocate for a comprehensive AI-augmented ecosystem, leveraging Large Language Models (LLMs) not as replacements for human judgment, but as sophisticated collaborators for authors, reviewers, and Area Chairs (ACs). We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making. Crucially, we contend that the development of such systems hinges on access to more granular, structured, and ethically-sourced peer review process data. We outline a research agenda, including illustrative experiments, to develop and validate these AI assistants, and discuss significant technical and ethical challenges. We call upon the ML community to proactively build this AI-assisted future, ensuring the continued integrity and scalability of scientific validation, while maintaining high standards of peer review.

📄 PDF Abstract BibTeX arXiv:2506.08134

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Is Peer-Reviewing Worth the Effort?

2024-12-18 · Kenneth Church, Raman Chandrasekar, John E. Ortega, Ibrahim Said Ahmad

How effective is peer-reviewing in identifying important papers? We treat this question as a forecasting task. Can we predict which papers will be highly cited in the future based on venue and "early returns" (citations …

Paper Quality Assessment based on Individual Wisdom Metrics from Open Peer Review

2025-01-22 · Andrii Zahorodnii, Jasper J. F. van den Bosch, Ian Charest, Christopher Summerfield 외

This study proposes a data-driven framework for enhancing the accuracy and efficiency of scientific peer review through an open, bottom-up process that estimates reviewer quality. Traditional closed peer review systems, …

Fairness

PeeriScope: A Multi-Faceted Framework for Evaluating Peer Review Quality

2026-04-27 · Sajad Ebrahimi, Soroush Sadeghian, Ali Ghorbanpour, Negar Arabzadeh 외 arxiv

The increasing scale and variability of peer review in scholarly venues has created an urgent need for systematic, interpretable, and extensible tools to assess review quality. We present PeeriScope, a modular platform t…

Auctions and Peer Prediction for Academic Peer Review

2021-08-27 · Siddarth Srinivasan, Jamie Morgenstern

Peer reviewed publications are considered the gold standard in certifying and disseminating ideas that a research community considers valuable. However, we identify two major drawbacks of the current system: (1) the over…

Prediction

PeerSum: A Peer Review Dataset for Abstractive Multi-document Summarization

2021-11-16 · ACL ARR November 2021 11 · Anonymous

We present PeerSum, a new MDS dataset using peer reviews of scientific publications. Our dataset differs from the existing MDS datasets in that our summaries (i.e., the meta-reviews) are highly abstractive and they are r…

Document SummarizationMulti-Document Summarization