Pitfalls and Best Practices in Algorithm Configuration
Good parameter settings are crucial to achieve high performance in many areas of artificial intelligence (AI), such as propositional satisfiability solving, AI planning, scheduling, and machine learning (in particular deep learning). Automated algorithm configuration methods have recently received much attention in the AI community since they replace tedious, irreproducible and error-prone manual parameter tuning and can lead to new state-of-the-art performance. However, practical applications of algorithm configuration are prone to several (often subtle) pitfalls in the experimental design that can render the procedure ineffective. We identify several common issues and propose best practices for avoiding them. As one possibility for automatically handling as many of these as possible, we also propose a tool called GenericWrapper4AC.
Code (2)
Tasks
Experimental DesignSchedulingSimilar Papers 제목 키워드 기반
Adapting Serious Game for Fallacious Argumentation to German: Pitfalls, Insights, and Best Practices
Considerations, Good Practices, Risks and Pitfalls in Developing AI Solutions Against COVID-19
The COVID-19 pandemic has been a major challenge to humanity, with 12.7 million confirmed cases as of July 13th, 2020 [1]. In previous work, we described how Artificial Intelligence can be used to tackle the pandemic wit…
IoT Device Identification with Machine Learning: Common Pitfalls and Best Practices
This paper critically examines the device identification process using machine learning, addressing common pitfalls in existing literature. We analyze the trade-offs between identification methods (unique vs. class based…
Data AugmentationTowards machine learning guided by best practices
Nowadays, machine learning (ML) is being used in software systems with multiple application fields, from medicine to software engineering (SE). On the one hand, the popularity of ML in the industry can be seen in the sta…
On Evaluating Adversarial Robustness of Chest X-ray Classification: Pitfalls and Best Practices
Vulnerability to adversarial attacks is a well-known weakness of Deep Neural Networks. While most of the studies focus on natural images with standardized benchmarks like ImageNet and CIFAR, little research has considere…
Adversarial RobustnessClassificationMedical DiagnosisX-ray Classification