What happens when reviewers receive AI feedback in their reviews?
AI is reshaping academic research, yet its role in peer review remains polarising and contentious. Advocates see its potential to reduce reviewer burden and improve quality, while critics warn of risks to fairness, accountability, and trust. At ICLR 2025, an official AI feedback tool was deployed to provide reviewers with post-review suggestions. We studied this deployment through surveys and interviews, investigating how reviewers engaged with the tool and perceived its usability and impact. Our findings surface both opportunities and tensions when AI augments in peer review. This work contributes the first empirical evidence of such an AI tool in a live review process, documenting how reviewers respond to AI-generated feedback in a high-stakes review context. We further offer design implications for AI-assisted reviewing that aim to enhance quality while safeguarding human expertise, agency, and responsibility.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
Peer review at AI conferences is stressed by rapidly rising submission volumes, leading to deteriorating review quality and increased author dissatisfaction. To address these issues, we developed Review Feedback Agent, a…
Good practices for evaluation of synthesized speech
This document is provided as a guideline for reviewers of papers about speech synthesis. We outline some best practices and common pitfalls for papers about speech synthesis, with a particular focus on evaluation. We als…
Speech SynthesisPrior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review
Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower r…
BIG-bench Machine LearningCan A User Anticipate What Her Followers Want?
Whenever a social media user decides to share a story, she is typically pleased to receive likes, comments, shares, or, more generally, feedback from her followers. As a result, she may feel compelled to use the feedback…
Decision MakingSequential Decision MakingTwo-sample testingBode Integral Limitation For Irrational Systems
Bode integrals of sensitivity and sensitivity-like functions along with complementary sensitivity and complementary sensitivity-like functions are conventionally used for describing performance limitations of a feedback …
Sensitivity