Enhancing Peer Review in Astronomy: A Machine Learning and Optimization Approach to Reviewer Assignments for ALMA
The increasing volume of papers and proposals that undergo peer review emphasizes the pressing need for greater automation to effectively manage the growing scale. In this study, we present the deployment and evaluation of machine learning and optimization techniques to assign proposals to reviewers that were developed for the Atacama Large Millimeter/submillimeter Array (ALMA) during the Cycle 10 Call for Proposals issued in 2023. Using topic modeling algorithms, we identify the proposal topics and assess reviewers' expertise based on their previous ALMA proposal submissions. We then apply an adapted version of the assignment optimization algorithm from PeerReview4All (Stelmakh et al. 2021) to maximize the alignment between proposal topics and reviewer expertise. Our evaluation shows a significant improvement in matching reviewer expertise: the median similarity score between the proposal topic and reviewer expertise increased by 51 percentage points compared to the previous cycle, and the percentage of reviewers reporting expertise in their assigned proposals rose by 20 percentage points. Furthermore, the assignment process proved highly effective in that no proposals required reassignment due to significant mismatches, resulting in a savings of 3 to 5 days of manual effort.
Code (0)
등록된 구현이 없습니다.
Tasks
AstronomySimilar Papers 제목 키워드 기반
pathfinder: A Semantic Framework for Literature Review and Knowledge Discovery in Astronomy
The exponential growth of astronomical literature poses significant challenges for researchers navigating and synthesizing general insights or even domain-specific knowledge. We present Pathfinder, a machine learning fra…
AstronomyPathfinderThe AI Imperative: Scaling High-Quality Peer Review in Machine Learning
Peer review, the bedrock of scientific advancement in machine learning (ML), is strained by a crisis of scale. Exponential growth in manuscript submissions to premier ML venues such as NeurIPS, ICML, and ICLR is outpacin…
AstroReview: An LLM-driven Multi-Agent Framework for Telescope Proposal Peer Review and Refinement
Competitive access to modern observatories has intensified as proposal volumes outpace available telescope time, making timely, consistent, and transparent peer review a critical bottleneck for the advancement of astrono…
Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions
Large Language Models (LLMs) have demonstrated wide-ranging applications across various fields and have shown significant potential in the academic peer-review process. However, existing applications are primarily limite…
Review GenerationPaper Quality Assessment based on Individual Wisdom Metrics from Open Peer Review
This study proposes a data-driven framework for enhancing the accuracy and efficiency of scientific peer review through an open, bottom-up process that estimates reviewer quality. Traditional closed peer review systems, …
Fairness