Automatic Analysis of Available Source Code of Top Artificial Intelligence Conference Papers
Source code is essential for researchers to reproduce the methods and replicate the results of artificial intelligence (AI) papers. Some organizations and researchers manually collect AI papers with available source code to contribute to the AI community. However, manual collection is a labor-intensive and time-consuming task. To address this issue, we propose a method to automatically identify papers with available source code and extract their source code repository URLs. With this method, we find that 20.5% of regular papers of 10 top AI conferences published from 2010 to 2019 are identified as papers with available source code and that 8.1% of these source code repositories are no longer accessible. We also create the XMU NLP Lab README Dataset, the largest dataset of labeled README files for source code document research. Through this dataset, we have discovered that quite a few README files have no installation instructions or usage tutorials provided. Further, a large-scale comprehensive statistical analysis is made for a general picture of the source code of AI conference papers. The proposed solution can also go beyond AI conference papers to analyze other scientific papers from both journals and conferences to shed light on more domains.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
HENNC: Hardware Engine for Artificial Neural Network-based Chaotic Oscillators
This letter introduces a framework for the automatic generation of hardware cores for Artificial Neural Network (ANN)-based chaotic oscillators. The framework trains the model to approximate a chaotic system, then perfor…
High-Level SynthesiseSCAPE: a Large-scale Synthetic Corpus for Automatic Post-Editing
Training models for the automatic correction of machine-translated text usually relies on data consisting of (source, MT, human post- edit) triplets providing, for each source sentence, examples of translation errors wit…
Automatic Post-EditingSentenceTraining Effective Neural Sentence Encoders from Automatically Mined Paraphrases
Sentence embeddings are commonly used in text clustering and semantic retrieval tasks. State-of-the-art sentence representation methods are based on artificial neural networks fine-tuned on large collections of manually …
Language ModelingLanguage ModellingRetrievalSemantic Retrieval+3OpenMedIA: Open-Source Medical Image Analysis Toolbox and Benchmark under Heterogeneous AI Computing Platforms
In this paper, we present OpenMedIA, an open-source toolbox library containing a rich set of deep learning methods for medical image analysis under heterogeneous Artificial Intelligence (AI) computing platforms. Various …
image-classificationImage ClassificationMedical Image AnalysisMedical Image Classification+2Automatic classification of prostate MR series type using image content and metadata
With the wealth of medical image data, efficient curation is essential. Assigning the sequence type to magnetic resonance images is necessary for scientific studies and artificial intelligence-based analysis. However, in…