Papers Hyperparameter Optimization
“Hyperparameter Optimization” 태그가 달린 논문 1,012편 · 필터 해제
Mental Task Classification Using Electroencephalogram Signal
This paper studies the classification problem on electroencephalogram (EEG) data of mental tasks, using standard architecture of three-layer CNN, stacked LSTM, stacked GRU. We further propose a novel classifier - a mixed…
ClassificationDecoderEEGElectroencephalogram (EEG)+2A Quantile-based Approach for Hyperparameter Transfer Learning
Bayesian optimization (BO) is a popular methodology to tune the hyperparameters of expensive black-box functions. Traditionally, BO focuses on a single task at a time and is not designed to leverage information from rela…
Bayesian OptimizationHyperparameter OptimizationNeural Architecture SearchThompson Sampling+1Towards modular and programmable architecture search
Neural architecture search methods are able to find high performance deep learning architectures with minimal effort from an expert. However, current systems focus on specific use-cases (e.g. convolutional image classifi…
Hyperparameter OptimizationNeural Architecture SearchGradient Descent: The Ultimate Optimizer
Working with any gradient-based machine learning algorithm involves the tedious task of tuning the optimizer's hyperparameters, such as its step size. Recent work has shown how the step size can itself be optimized along…
BIG-bench Machine LearningHyperparameter OptimizationLearning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
Bayesian optimization (BO) is a successful methodology to optimize black-box functions that are expensive to evaluate. While traditional methods optimize each black-box function in isolation, there has been recent intere…
Bayesian OptimizationHyperparameter OptimizationTransfer LearningOn Federated Learning of Deep Networks from Non-IID Data: Parameter Divergence and the Effects of Hyperparametric Methods
Federated learning, where a global model is trained by iterative parameter averaging of locally-computed updates, is a promising approach for distributed training of deep networks; it provides high communication-efficien…
Federated LearningHyperparameter OptimizationScheduling the Learning Rate Via Hypergradients: New Insights and a New Algorithm
We study the problem of fitting task-specific learning rate schedules from the perspective of hyperparameter optimization. This allows us to explicitly search for schedules that achieve good generalization. We describe …
Hyperparameter OptimizationSchedulingTraining Deep Neural Networks by optimizing over nonlocal paths in hyperparameter space
Hyperparameter optimization is both a practical issue and an interesting theoretical problem in training of deep architectures. Despite many recent advances the most commonly used methods almost universally involve train…
Hyperparameter OptimizationA scalable constructive algorithm for the optimization of neural network architectures
We propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal…
Hyperparameter OptimizationTransferable Neural Processes for Hyperparameter Optimization
Automated machine learning aims to automate the whole process of machine learning, including model configuration. In this paper, we focus on automated hyperparameter optimization (HPO) based on sequential model-based opt…
BIG-bench Machine LearningHyperparameter OptimizationTransfer LearningRandom Error Sampling-based Recurrent Neural Network Architecture Optimization
Recurrent neural networks are good at solving prediction problems. However, finding a network that suits a problem is quite hard because their performance is strongly affected by their architecture configuration. Automat…
Hyperparameter OptimizationEnabling hyperparameter optimization in sequential autoencoders for spiking neural data
Continuing advances in neural interfaces have enabled simultaneous monitoring of spiking activity from hundreds to thousands of neurons. To interpret these large-scale data, several methods have been proposed to infer la…
Hyperparameter OptimizationTowards Assessing the Impact of Bayesian Optimization's Own Hyperparameters
Bayesian Optimization (BO) is a common approach for hyperparameter optimization (HPO) in automated machine learning. Although it is well-accepted that HPO is crucial to obtain well-performing machine learning models, tun…
Bayesian OptimizationBIG-bench Machine LearningHyperparameter OptimizationNeural Architecture SearchHybrid methodology based on Bayesian optimization and GA-PARSIMONY to search for parsimony models by combining hyperparameter optimization and feature selection
This article presents a hybrid methodology that combines Bayesian optimization (BO) with a constrained version of the GA-PARSIMONY method to obtain parsimony models. The proposal is designed to reduce the sizeable comput…
Bayesian Optimizationfeature selectionHyperparameter OptimizationModel SelectionBOAH: A Tool Suite for Multi-Fidelity Bayesian Optimization & Analysis of Hyperparameters
Hyperparameter optimization and neural architecture search can become prohibitively expensive for regular black-box Bayesian optimization because the training and evaluation of a single model can easily take several hour…
Bayesian OptimizationHyperparameter OptimizationNeural Architecture SearchTowards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tools
There has been considerable growth and interest in industrial applications of machine learning (ML) in recent years. ML engineers, as a consequence, are in high demand across the industry, yet improving the efficiency of…
AutoMLBIG-bench Machine LearningFeature EngineeringHyperparameter Optimization+1AutoML: A Survey of the State-of-the-Art
Deep learning (DL) techniques have penetrated all aspects of our lives and brought us great convenience. However, building a high-quality DL system for a specific task highly relies on human expertise, hindering the appl…
AutoMLFeature EngineeringHyperparameter OptimizationNeural Architecture Search+1Deep Neural Network Hyperparameter Optimization with Orthogonal Array Tuning
Deep learning algorithms have achieved excellent performance lately in a wide range of fields (e.g., computer version). However, a severe challenge faced by deep learning is the high dependency on hyper-parameters. The a…
Bayesian OptimizationDeep LearningHyperparameter OptimizationOptuna: A Next-generation Hyperparameter Optimization Framework
The purpose of this study is to introduce new design-criteria for next-generation hyperparameter optimization software. The criteria we propose include (1) define-by-run API that allows users to construct the parameter s…
Distributed ComputingHyperparameter OptimizationSpectral Overlap and a Comparison of Parameter-Free, Dimensionality Reduction Quality Metrics
Nonlinear dimensionality reduction methods are a popular tool for data scientists and researchers to visualize complex, high dimensional data. However, while these methods continue to improve and grow in number, it is of…
Dimensionality ReductionHyperparameter Optimization