Avoiding a Tragedy of the Commons in the Peer Review Process
Peer review is the foundation of scientific publication, and the task of reviewing has long been seen as a cornerstone of professional service. However, the massive growth in the field of machine learning has put this community benefit under stress, threatening both the sustainability of an effective review process and the overall progress of the field. In this position paper, we argue that a tragedy of the commons outcome may be avoided by emphasizing the professional aspects of this service. In particular, we propose a rubric to hold reviewers to an objective standard for review quality. In turn, we also propose that reviewers be given appropriate incentive. As one possible such incentive, we explore the idea of financial compensation on a per-review basis. We suggest reasonable funding models and thoughts on long term effects.
Code (0)
등록된 구현이 없습니다.
Tasks
PositionSimilar Papers 제목 키워드 기반
Beyond the Tragedy of the Commons: Building A Reputation System for Generative Multi-agent Systems
The tragedy of the commons, where individual self-interest leads to collectively disastrous outcomes, is a pervasive challenge in human society. Recent studies have demonstrated that similar phenomena can arise in genera…
Artificial virtuous agents in a multiagent tragedy of the commons
Although virtue ethics has repeatedly been proposed as a suitable framework for the development of artificial moral agents (AMAs), it has been proven difficult to approach from a computational perspective. In this work, …
EthicsHypochaos prevents tragedy of the commons in discrete-time eco-evolutionary game dynamics
While quite a few recent papers have explored game-resource feedback using the framework of evolutionary game theory, almost all the studies are confined to using time-continuous dynamical equations. Moreover, in such li…
Seven rules to avoid the tragedy of the commons
Cooperation among self-interested players in a social dilemma is fragile and easily interrupted by mistakes. In this work, we study the repeated $n$-person public-goods game and search for a strategy that forms a coopera…
Learning from Committee: Reasoning Distillation from a Mixture of Teachers with Peer-Review
While reasoning capabilities typically emerge in large language models (LLMs) with tens of billions of parameters, recent research focuses on improving smaller open-source models through knowledge distillation (KD) from …
Knowledge DistillationLogical Reasoning