Is Peer-Reviewing Worth the Effort?
How effective is peer-reviewing in identifying important papers? We treat this question as a forecasting task. Can we predict which papers will be highly cited in the future based on venue and "early returns" (citations soon after publication)? We show early returns are more predictive than venue. Finally, we end with constructive suggestions to address scaling challenges: (a) too many submissions and (b) too few qualified reviewers.
Code (1)
Similar Papers 제목 키워드 기반
What Can Natural Language Processing Do for Peer Review?
The number of scientific articles produced every year is growing rapidly. Providing quality control over them is crucial for scientists and, ultimately, for the public good. In modern science, this process is largely del…
ArticlesAuctions and Peer Prediction for Academic Peer Review
Peer reviewed publications are considered the gold standard in certifying and disseminating ideas that a research community considers valuable. However, we identify two major drawbacks of the current system: (1) the over…
PredictionDouble blind vs. open review: an evolutionary game logit-simulating the behavior of authors and reviewers
Despite the tremendous successes of science in providing knowledge and technologies, the Replication Crisis has highlighted that scientific institutions have much room for improvement. Peer-review is one target of critic…
Gen-Review: A Large-scale Dataset of AI-Generated (and Human-written) Peer Reviews
How does the progressive embracement of Large Language Models (LLMs) affect scientific peer reviewing? This multifaceted question is fundamental to the effectiveness -- as well as to the integrity -- of the scientific pr…
NLPeer: A Unified Resource for the Computational Study of Peer Review
Peer review constitutes a core component of scholarly publishing; yet it demands substantial expertise and training, and is susceptible to errors and biases. Various applications of NLP for peer reviewing assistance aim …