Papers Hyperparameter Optimization
“Hyperparameter Optimization” 태그가 달린 논문 1,012편 · 필터 해제
Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS
In this paper, we compare the three most popular algorithms for hyperparameter optimization (Grid Search, Random Search, and Genetic Algorithm) and attempt to use them for neural architecture search (NAS). We use these a…
Hyperparameter OptimizationNeural Architecture SearchSequential vs. Integrated Algorithm Selection and Configuration: A Case Study for the Modular CMA-ES
When faced with a specific optimization problem, choosing which algorithm to use is always a tough task. Not only is there a vast variety of algorithms to select from, but these algorithms often are controlled by many hy…
global-optimizationHyperparameter OptimizationBayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax
Deep learning models are full of hyperparameters, which are set manually before the learning process can start. To find the best configuration for these hyperparameters in such a high dimensional space, with time-consumi…
Bayesian OptimizationGaussian ProcessesGraph ClassificationHyperparameter OptimizationSimpler Hyperparameter Optimization for Software Analytics: Why, How, When?
How can we make software analytics simpler and faster? One method is to match the complexity of analysis to the intrinsic complexity of the data being explored. For example, hyperparameter optimizers find the control set…
Hyperparameter OptimizationImproved Covariance Matrix Estimator using Shrinkage Transformation and Random Matrix Theory
One of the major challenges in multivariate analysis is the estimation of population covariance matrix from sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or…
Hyperparameter OptimizationExperienceThinking: Constrained Hyperparameter Optimization based on Knowledge and Pruning
Machine learning algorithms are very sensitive to the hyperparameters, and their evaluations are generally expensive. Users desperately need intelligent methods to quickly optimize hyperparameter settings according to kn…
BIG-bench Machine LearningHyperparameter OptimizationNeural Architecture SearchSingle Headed Attention RNN: Stop Thinking With Your Head
The leading approaches in language modeling are all obsessed with TV shows of my youth - namely Transformers and Sesame Street. Transformers this, Transformers that, and over here a bonfire worth of GPU-TPU-neuromorphic …
GPUHyperparameter OptimizationLanguage ModelingLanguage ModellingA Simple Heuristic for Bayesian Optimization with A Low Budget
The aim of black-box optimization is to optimize an objective function within the constraints of a given evaluation budget. In this problem, it is generally assumed that the computational cost for evaluating a point is l…
Bayesian OptimizationHyperparameter OptimizationA Neural Network Based on the Johnson $S_\mathrm{U}$ Translation System and Related Application to Electromyogram Classification
Electromyogram (EMG) classification is a key technique in EMG-based control systems. The existing EMG classification methods do not consider the characteristics of EMG features that the distribution has skewness and kurt…
ClassificationGeneral ClassificationHyperparameter OptimizationTranslationFLAML: A Fast and Lightweight AutoML Library
We study the problem of using low computational cost to automate the choices of learners and hyperparameters for an ad-hoc training dataset and error metric, by conducting trials of different configurations on the given …
AutoMLHyperparameter OptimizationConstructing Gradient Controllable Recurrent Neural Networks Using Hamiltonian Dynamics
Recurrent neural networks (RNNs) have gained a great deal of attention in solving sequential learning problems. The learning of long-term dependencies, however, remains challenging due to the problem of a vanishing or ex…
Hyperparameter OptimizationAuptimizer -- an Extensible, Open-Source Framework for Hyperparameter Tuning
Tuning machine learning models at scale, especially finding the right hyperparameter values, can be difficult and time-consuming. In addition to the computational effort required, this process also requires some ancillar…
ARCBIG-bench Machine LearningHyperparameter OptimizationNeural Architecture Search+1Optimizing Millions of Hyperparameters by Implicit Differentiation
We propose an algorithm for inexpensive gradient-based hyperparameter optimization that combines the implicit function theorem (IFT) with efficient inverse Hessian approximations. We present results about the relationshi…
Data AugmentationHyperparameter OptimizationPrior Specification for Bayesian Matrix Factorization via Prior Predictive Matching
The behavior of many Bayesian models used in machine learning critically depends on the choice of prior distributions, controlled by some hyperparameters that are typically selected by Bayesian optimization or cross-vali…
Bayesian OptimizationHyperparameter OptimizationStochastic OptimizationBANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search
Over the past half-decade, many methods have been considered for neural architecture search (NAS). Bayesian optimization (BO), which has long had success in hyperparameter optimization, has recently emerged as a very pro…
Bayesian OptimizationHyperparameter OptimizationNeural Architecture SearchReinforcement LearningAuto-Model: Utilizing Research Papers and HPO Techniques to Deal with the CASH problem
In many fields, a mass of algorithms with completely different hyperparameters have been developed to address the same type of problems. Choosing the algorithm and hyperparameter setting correctly can promote the overall…
Hyperparameter OptimizationMARTHE: Scheduling the Learning Rate Via Online Hypergradients
We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization, aiming at good generalization. We describe the structure of the gradient of a validation error w.…
Hyperparameter OptimizationSchedulingAnatomically-Informed Data Augmentation for functional MRI with Applications to Deep Learning
The application of deep learning to build accurate predictive models from functional neuroimaging data is often hindered by limited dataset sizes. Though data augmentation can help mitigate such training obstacles, most …
Data AugmentationHyperparameter OptimizationConstrained Bayesian Optimization with Max-Value Entropy Search
Bayesian optimization (BO) is a model-based approach to sequentially optimize expensive black-box functions, such as the validation error of a deep neural network with respect to its hyperparameters. In many real-world s…
Bayesian OptimizationHyperparameter OptimizationProbabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings
We propose probabilistic models that can extrapolate learning curves of iterative machine learning algorithms, such as stochastic gradient descent for training deep networks, based on training data with variable-length l…
BIG-bench Machine LearningHyperparameter Optimization