OPTION: OPTImization Algorithm Benchmarking ONtology
Many platforms for benchmarking optimization algorithms offer users the possibility of sharing their experimental data with the purpose of promoting reproducible and reusable research. However, different platforms use different data models and formats, which drastically inhibits identification of relevant data sets, their interpretation, and their interoperability. Consequently, a semantically rich, ontology-based, machine-readable data model is highly desired. We report in this paper on the development of such an ontology, which we name OPTION (OPTImization algorithm benchmarking ONtology). Our ontology provides the vocabulary needed for semantic annotation of the core entities involved in the benchmarking process, such as algorithms, problems, and evaluation measures. It also provides means for automated data integration, improved interoperability, powerful querying capabilities and reasoning, thereby enriching the value of the benchmark data. We demonstrate the utility of OPTION by annotating and querying a corpus of benchmark performance data from the BBOB workshop data - a use case which can be easily extended to cover other benchmarking data collections.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingData IntegrationSimilar Papers 제목 키워드 기반
OPTION: OPTImization Algorithm Benchmarking ONtology
Many optimization algorithm benchmarking platforms allow users to share their experimental data to promote reproducible and reusable research. However, different platforms use different data models and formats, which dra…
BenchmarkingData IntegrationUsing Knowledge Graphs for Performance Prediction of Modular Optimization Algorithms
Empirical data plays an important role in evolutionary computation research. To make better use of the available data, ontologies have been proposed in the literature to organize their storage in a structured way. Howeve…
Graph EmbeddingKnowledge Graph EmbeddingKnowledge GraphsTriple ClassificationIOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics
We present IOHexperimenter, the experimentation module of the IOHprofiler project, which aims at providing an easy-to-use and highly customizable toolbox for benchmarking iterative optimization heuristics such as local s…
Bayesian OptimizationBenchmarkingGENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles
Recent works in Event Argument Extraction (EAE) have focused on improving model generalizability to cater to new events and domains. However, standard benchmarking datasets like ACE and ERE cover less than 40 event types…
BenchmarkingEvent Argument ExtractionSemantic Role LabelingSlot Filling for Extracting Reskilling and Upskilling Options from the Web
Disturbances in the job market such as advances in science and technology, crisis and increased competition have triggered a surge in reskilling and upskilling programs. Information on suitable continuing education optio…
BenchmarkingEntity Linkingslot-fillingSlot Filling