Building benchmarking frameworks for supporting replicability and reproducibility: spatial and textual analysis as an example
Replicability and reproducibility (R&R) are critical for the long-term prosperity of a scientific discipline. In GIScience, researchers have discussed R&R related to different research topics and problems, such as local spatial statistics, digital earth, and metadata (Fotheringham, 2009; Goodchild, 2012; Anselin et al., 2014). This position paper proposes to further support R&R by building benchmarking frameworks in order to facilitate the replication of previous research for effective and effcient comparisons of methods and software tools developed for addressing the same or similar problems. Particularly, this paper will use geoparsing, an important research problem in spatial and textual analysis, as an example to explain the values of such benchmarking frameworks.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingPositionSimilar Papers 제목 키워드 기반
ReproGen: Proposal for a Shared Task on Reproducibility of Human Evaluations in NLG
Across NLP, a growing body of work is looking at the issue of reproducibility. However, replicability of human evaluation experiments and reproducibility of their results is currently under-addressed, and this is of part…
A Study on Reproducibility and Replicability of Table Structure Recognition Methods
Concerns about reproducibility in artificial intelligence (AI) have emerged, as researchers have reported unsuccessful attempts to directly reproduce published findings in the field. Replicability, the ability to affirm …
CISOL: An Open and Extensible Dataset for Table Structure Recognition in the Construction Industry
Reproducibility and replicability are critical pillars of empirical research, particularly in machine learning, where they depend not only on the availability of models, but also on the datasets used to train and evaluat…
BenchmarkingObject DetectionTable DetectionTable ExtractionStore Scientific Workflows Data in SSHOC Repository
Today scientific workflows are used by scientists as a way to define automated, scalable, and portable in-silico experiments. Having a formal description of an experiment can improve replicability and reproducibility of …
ManagementGeoAI Reproducibility and Replicability: a computational and spatial perspective
GeoAI has emerged as an exciting interdisciplinary research area that combines spatial theories and data with cutting-edge AI models to address geospatial problems in a novel, data-driven manner. While GeoAI research has…