ExperienceThinking: Constrained Hyperparameter Optimization based on Knowledge and Pruning
Machine learning algorithms are very sensitive to the hyperparameters, and their evaluations are generally expensive. Users desperately need intelligent methods to quickly optimize hyperparameter settings according to known evaluation information, and thus reduce computational cost and promote optimization efficiency. Motivated by this, we propose ExperienceThinking algorithm to quickly find the best possible hyperparameter configuration of machine learning algorithms within a few configuration evaluations. ExperienceThinking design two novel methods, which intelligently infer optimal configurations from two aspects: search space pruning and knowledge utilization respectively. Two methods complement each other and solve the constrained hyperparameter optimization problems effectively. To demonstrate the benefit of ExperienceThinking, we compare it with 3 classical hyperparameter optimization algorithms with a small number of configuration evaluations. The experimental results present that our proposed algorithm provides superior results and achieve better performance.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningHyperparameter OptimizationNeural Architecture SearchMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Combined Pruning for Nested Cross-Validation to Accelerate Automated Hyperparameter Optimization for Embedded Feature Selection in High-Dimensional Data with Very Small Sample Sizes
Background: Embedded feature selection in high-dimensional data with very small sample sizes requires optimized hyperparameters for the model building process. For this hyperparameter optimization, nested cross-validatio…
feature selectionHyperparameter OptimizationHyperparameter Optimization with Neural Network Pruning
Since the deep learning model is highly dependent on hyperparameters, hyperparameter optimization is essential in developing deep learning model-based applications, even if it takes a long time. As service development us…
Bayesian OptimizationDeep LearningHyperparameter OptimizationNetwork PruningOnline Convex Optimization with Unconstrained Domains and Losses
We propose an online convex optimization algorithm (RescaledExp) that achieves optimal regret in the unconstrained setting without prior knowledge of any bounds on the loss functions. We prove a lower bound showing an ex…
Hyperparameter OptimizationQuantity vs. Quality: On Hyperparameter Optimization for Deep Reinforcement Learning
Reinforcement learning algorithms can show strong variation in performance between training runs with different random seeds. In this paper we explore how this affects hyperparameter optimization when the goal is to find…
Bayesian OptimizationDeep Reinforcement LearningHyperparameter Optimizationreinforcement-learning+2Pruning-Based TinyML Optimization of Machine Learning Models for Anomaly Detection in Electric Vehicle Charging Infrastructure
With the growing need for real-time processing on IoT devices, optimizing machine learning (ML) models' size, latency, and computational efficiency is essential. This paper investigates a pruning method for anomaly detec…
Anomaly DetectionComputational Efficiencyfeature selection