The Great AI Witch Hunt: Reviewers Perception and (Mis)Conception of Generative AI in Research Writing
Generative AI (GenAI) use in research writing is growing fast. However, it is unclear how peer reviewers recognize or misjudge AI-augmented manuscripts. To investigate the impact of AI-augmented writing on peer reviews, we conducted a snippet-based online survey with 17 peer reviewers from top-tier HCI conferences. Our findings indicate that while AI-augmented writing improves readability, language diversity, and informativeness, it often lacks research details and reflective insights from authors. Reviewers consistently struggled to distinguish between human and AI-augmented writing but their judgements remained consistent. They noted the loss of a "human touch" and subjective expressions in AI-augmented writing. Based on our findings, we advocate for reviewer guidelines that promote impartial evaluations of submissions, regardless of any personal biases towards GenAI. The quality of the research itself should remain a priority in reviews, regardless of any preconceived notions about the tools used to create it. We emphasize that researchers must maintain their authorship and control over the writing process, even when using GenAI's assistance.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityInformativenessSimilar Papers 제목 키워드 기반
Apologia Pro Vita Sua: The Vanishing of the White Whale in the Mists
There are many analogies among fortune hunting in business, politics, and science. The prime task of the gold digger was to go to the Klondikes, find the right mine and mine the richest veins. This task requires motivati…
Individual vs. Joint Perception: a Pragmatic Model of Pointing as Communicative Smithian Helping
The simple gesture of pointing can greatly augment ones ability to comprehend states of the world based on observations. It triggers additional inferences relevant to ones task at hand. We model an agents update to its b…
Who Reviews The Reviewers? A Multi-Level Jury Problem
We consider the problem of determining a binary ground truth using advice from a group of independent reviewers (experts) who express their guess about a ground truth correctly with some independent probability (competen…
ScholarPeer: A Context-Aware Multi-Agent Framework for Automated Peer Review
The exponential growth of machine learning submissions has strained the traditional peer review process, resulting in slow feedback loops for authors and an immense burden on reviewers to rigorously audit technical sound…
DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing
In company with the data explosion over the past decade, deep neural network (DNN) based software has experienced unprecedented leap and is becoming the key driving force of many novel industrial applications, including …
Autonomous DrivingQuantization