A Shared Task of a New, Collaborative Type to Foster Reproducibility: A First Exercise in the Area of Language Science and Technology with REPROLANG2020
n this paper, we introduce a new type of shared task {---} which is collaborative rather than competitive {---} designed to support and fosterthe reproduction of research results. We also describe the first event running such a novel challenge, present the results obtained, discussthe lessons learned and ponder on future undertakings.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
The ReproGen Shared Task on Reproducibility of Human Evaluations in NLG: Overview and Results
The NLP field has recently seen a substantial increase in work related to reproducibility of results, and more generally in recognition of the importance of having shared definitions and practices relating to evaluation.…
Predicting computational reproducibility of data analysis pipelines in large population studies using collaborative filtering
Evaluating the computational reproducibility of data analysis pipelines has become a critical issue. It is, however, a cumbersome process for analyses that involve data from large populations of subjects, due to their co…
Collaborative FilteringAn End-to-End System for Reproducibility Assessment of Source Code Repositories via Their Readmes
Increased reproducibility of machine learning research has been a driving force for dramatic improvements in learning performances. The scientific community further fosters this effort by including reproducibility rating…
Evaluating the method reproducibility of deep learning models in the biodiversity domain
Artificial Intelligence (AI) is revolutionizing biodiversity research by enabling advanced data analysis, species identification, and habitats monitoring, thereby enhancing conservation efforts. Ensuring reproducibility …
Deep LearningMACC: Multi-Agent Collaborative Competition for Scientific Exploration
Scientific discovery still relies heavily on the manual efforts of individual researchers, leading to limited exploration, redundant trials, and reduced reproducibility. Human-participant data analysis competitions gener…