Yes-Yes-Yes: Proactive Data Collection for ACL Rolling Review and Beyond
The shift towards publicly available text sources has enabled language processing at unprecedented scale, yet leaves under-serviced the domains where public and openly licensed data is scarce. Proactively collecting text data for research is a viable strategy to address this scarcity, but lacks systematic methodology taking into account the many ethical, legal and confidentiality-related aspects of data collection. Our work presents a case study on proactive data collection in peer review -- a challenging and under-resourced NLP domain. We outline ethical and legal desiderata for proactive data collection and introduce "Yes-Yes-Yes", the first donation-based peer reviewing data collection workflow that meets these requirements. We report on the implementation of Yes-Yes-Yes at ACL Rolling Review and empirically study the implications of proactive data collection for the dataset size and the biases induced by the donation behavior on the peer reviewing platform.
Code (1)
Similar Papers 제목 키워드 기반
From Passive Generation to Investigation: A Proactive Scientific Peer Review Agent
Large language models (LLMs) have shown promise in automating scientific peer review. However, existing approaches often struggle to generate in-depth reviews supported by concrete evidence. We argue that a key limitatio…
Reinforcement LearningBERT-Beta: A Proactive Probabilistic Approach to Text Moderation
Text moderation for user generated content, which helps to promote healthy interaction among users, has been widely studied and many machine learning models have been proposed. In this work, we explore an alternative per…
regressionIn Silico Approaches to Deliver Better Antibodies by Design: The Past, the Present and the Future
The recognition of the importance of drug-like properties beyond potency to reduce clinical attrition of biologics has driven significant progress in the development of in vitro and in silico tools for developability ass…
Drug DesignProactive Prioritization of App Issues via Contrastive Learning
Mobile app stores produce a tremendous amount of data in the form of user reviews, which is a huge source of user requirements and sentiments; such reviews allow app developers to proactively address issues in their apps…
Contrastive LearningAI for Anticipatory Action: Moving Beyond Climate Forecasting
Disaster response agencies have been shifting from a paradigm of climate forecasting towards one of anticipatory action: assessing not just what the climate will be, but how it will impact specific populations, thereby e…
Disaster Response