What Can We Do to Improve Peer Review in NLP?
Peer review is our best tool for judging the quality of conference submissions, but it is becoming increasingly spurious. We argue that a part of the problem is that the reviewers and area chairs face a poorly defined task forcing apples-to-oranges comparisons. There are several potential ways forward, but the key difficulty is creating the incentives and mechanisms for their consistent implementation in the NLP community.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
DeepSentiPeer: Harnessing Sentiment in Review Texts to Recommend Peer Review Decisions
Automatically validating a research artefact is one of the frontiers in Artificial Intelligence (AI) that directly brings it close to competing with human intellect and intuition. Although criticised sometimes, the exist…
What Factors Should Paper-Reviewer Assignments Rely On? Community Perspectives on Issues and Ideals in Conference Peer-Review
Both scientific progress and individual researcher careers depend on the quality of peer review, which in turn depends on paper-reviewer matching. Surprisingly, this problem has been mostly approached as an automated rec…
What Can Natural Language Processing Do for Peer Review?
The number of scientific articles produced every year is growing rapidly. Providing quality control over them is crucial for scientists and, ultimately, for the public good. In modern science, this process is largely del…
ArticlesWhat happens when reviewers receive AI feedback in their reviews?
AI is reshaping academic research, yet its role in peer review remains polarising and contentious. Advocates see its potential to reduce reviewer burden and improve quality, while critics warn of risks to fairness, accou…
Polarity in the Classroom: A Case Study Leveraging Peer Sentiment Toward Scalable Assessment
Accurately grading open-ended assignments in large or massive open online courses (MOOCs) is non-trivial. Peer review is a promising solution but can be unreliable due to few reviewers and an unevaluated review form. To …
Aspect ExtractionFormSentiment Analysis