Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search
While existing work on neural architecture search (NAS) tunes hyperparameters in a separate post-processing step, we demonstrate that architectural choices and other hyperparameter settings interact in a way that can render this separation suboptimal. Likewise, we demonstrate that the common practice of using very few epochs during the main NAS and much larger numbers of epochs during a post-processing step is inefficient due to little correlation in the relative rankings for these two training regimes. To combat both of these problems, we propose to use a recent combination of Bayesian optimization and Hyperband for efficient joint neural architecture and hyperparameter search.
Code (3)
Tasks
Bayesian OptimizationNeural Architecture SearchMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Joint Neural Architecture and Hyperparameter Search for Correlated Time Series Forecasting
Sensors in cyber-physical systems often capture interconnected processes and thus emit correlated time series (CTS), the forecasting of which enables important applications. The key to successful CTS forecasting is to un…
Correlated Time Series ForecastingTime SeriesTime Series AnalysisTime Series ForecastingEfficient and Joint Hyperparameter and Architecture Search for Collaborative Filtering
Automated Machine Learning (AutoML) techniques have recently been introduced to design Collaborative Filtering (CF) models in a data-specific manner. However, existing works either search architectures or hyperparameters…
AutoMLCollaborative FilteringBag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization
Neural architecture search (NAS) and hyperparameter optimization (HPO) make deep learning accessible to non-experts by automatically finding the architecture of the deep neural network to use and tuning the hyperparamete…
AutoMLHyperparameter OptimizationNeural Architecture SearchAutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification
Deep neural networks are powerful predictors for a variety of tasks. However, they do not capture uncertainty directly. Using neural network ensembles to quantify uncertainty is competitive with approaches based on Bayes…
Uncertainty QuantificationEfficient Automated Deep Learning for Time Series Forecasting
Recent years have witnessed tremendously improved efficiency of Automated Machine Learning (AutoML), especially Automated Deep Learning (AutoDL) systems, but recent work focuses on tabular, image, or NLP tasks. So far, l…
AutoMLBayesian OptimizationDeep LearningNeural Architecture Search+3