Best Practices For Empirical Meta-Algorithmic Research: Guidelines from the COSEAL Research Network
Empirical research on meta-algorithmics, such as algorithm selection, configuration, and scheduling, often relies on extensive and thus computationally expensive experiments. With the large degree of freedom we have over our experimental setup and design comes a plethora of possible error sources that threaten the scalability and validity of our scientific insights. Best practices for meta-algorithmic research exist, but they are scattered between different publications and fields, and continue to evolve separately from each other. In this report, we collect good practices for empirical meta-algorithmic research across the subfields of the COSEAL community, encompassing the entire experimental cycle: from formulating research questions and selecting an experimental design, to executing experiments, and ultimately, analyzing and presenting results impartially. It establishes the current state-of-the-art practices within meta-algorithmic research and serves as a guideline to both new researchers and practitioners in meta-algorithmic fields.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Meta-analysis of the relationship between quality management practices and firm performance—implications for quality management theory development
Quality management (QM) has received a high degree of attention in extant literature. Several research papers attribute superior firm performance to adoption of QM practices. The availability of a large number of resear…
AttributeManagementMetaLint: Easy-to-Hard Generalization for Code Linting
Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practices beyond those observed during training. We introduce MetaLint, a meta-learni…
Code GenerationSemi-Supervised Offline Reinforcement Learning with Action-Free Trajectories
Natural agents can effectively learn from multiple data sources that differ in size, quality, and types of measurements. We study this heterogeneity in the context of offline reinforcement learning (RL) by introducing a …
D4RLOffline RLreinforcement-learningReinforcement Learning+1Transfer Learning via Learning to Transfer
In transfer learning, what and how to transfer are two primary issues to be addressed, as different transfer learning algorithms applied between a source and a target domain result in different knowledge transferred…
Transfer LearningAssessing Phenotype Definitions for Algorithmic Fairness
Disease identification is a core, routine activity in observational health research. Cohorts impact downstream analyses, such as how a condition is characterized, how patient risk is defined, and what treatments are stud…
Fairness