Dynamic Design of Machine Learning Pipelines via Metalearning
Automated machine learning (AutoML) has democratized the design of machine learning based systems, by automating model selection, hyperparameter tuning and feature engineering. However, the high computational cost associated with traditional search and optimization strategies, such as Random Search, Particle Swarm Optimization and Bayesian Optimization, remains a significant challenge. Moreover, AutoML systems typically explore a large search space, which can lead to overfitting. This paper introduces a metalearning method for dynamically designing search spaces for AutoML system. The proposed method uses historical metaknowledge to select promising regions of the search space, accelerating the optimization process. According to experiments conducted for this study, the proposed method can reduce runtime by 89\% in Random Search and search space by (1.8/13 preprocessor and 4.3/16 classifier), without compromising significant predictive performance. Moreover, the proposed method showed competitive performance when adapted to Auto-Sklearn, reducing its search space. Furthermore, this study encompasses insights into meta-feature selection, meta-model explainability, and the trade-offs inherent in search space reduction strategies.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature EngineeringSimilar Papers 제목 키워드 기반
Preprocessor Selection for Machine Learning Pipelines
Much of the work in metalearning has focused on classifier selection, combined more recently with hyperparameter optimization, with little concern for data preprocessing. Yet, it is generally well accepted that machine l…
BIG-bench Machine LearningHyperparameter OptimizationA Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEs
Physics-informed neural networks (PINNs) as a means of discretizing partial differential equations (PDEs) are garnering much attention in the Computational Science and Engineering (CS&E) world. At least two challenges ex…
BIG-bench Machine LearningPhysics-informed machine learningTransfer LearningMetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translation
Manipulating training data leads to robust neural models for MT.
Machine TranslationTranslationRapid Adaptation with Conditionally Shifted Neurons
We describe a mechanism by which artificial neural networks can learn rapid adaptation - the ability to adapt on the fly, with little data, to new tasks - that we call conditionally shifted neurons. We apply this mechani…
Few-Shot Image ClassificationMetalearning for Feature Selection
A general formulation of optimization problems in which various candidate solutions may use different feature-sets is presented, encompassing supervised classification, automated program learning and other cases. A novel…
Classificationfeature selectionGeneral Classificationtext-classification+1