Commitment Checklist: Auditing Author Commitments in Peer Review
Peer review author responses often include commitments to add experiments, release code, or clarify content in the final paper. Yet, there is currently no systematic mechanism to ensure authors fulfill these promises. In this position paper, we present a large-scale audit of author commitments using large language models (LLMs) to compare rebuttals against camera-ready versions. Analyzing the commitments from ICLR-2025 and EMNLP-2024, we find that while a majority of promised changes are implemented, a significant share (about 25%) are not, with "missing experiments" and other high-impact items among the most frequently unfulfilled. We demonstrate that LLM-based tools can feasibly detect the promises. Finally, we propose the idea of Author Commitment Checklist, which would alert authors and organizers to unaddressed promises, increasing accountability and strengthening the integrity of the peer review process. We discuss the benefits of this practice and advocate for its adoption in future conferences.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Discourse vs emissions: Analysis of corporate narratives, symbolic practices, and mimicry through LLMs
Climate change has increased demands for transparent and comparable corporate climate disclosures, yet imitation and symbolic reporting often undermine their value. This paper develops a multidimensional framework to ass…
Consistency Ensuring in Social Web Services Based on Commitments Structure
Web Service is one of the most significant current discussions in information sharing technologies and one of the examples of service oriented processing. To ensure accurate execution of web services operations, it must …
From Hesitancy Framings to Vaccine Hesitancy Profiles: A Journey of Stance, Ontological Commitments and Moral Foundations
While billions of COVID-19 vaccines have been administered, too many people remain hesitant. Twitter, with its substantial reach and daily exposure, is an excellent resource for examining how people frame their vaccine h…
Do AI Companies Make Good on Voluntary Commitments to the White House?
Voluntary commitments are central to international AI governance, as demonstrated by recent voluntary guidelines from the White House to the G7, from Bletchley Park to Seoul. How do major AI companies make good on their …
Scaling up the discovery of hesitancy profiles by identifying the framing of beliefs towards vaccine confidence in Twitter discourse
Our study focused on the discovery of how vaccine hesitancy is framed in Twitter discourse, allowing us to recognize at-scale all tweets that evoke any of the hesitancy framings as well as the stance of the tweet authors…
Misinformation