Random Error Sampling-based Recurrent Neural Network Architecture Optimization
Recurrent neural networks are good at solving prediction problems. However, finding a network that suits a problem is quite hard because their performance is strongly affected by their architecture configuration. Automatic architecture optimization methods help to find the most suitable design, but they are not extensively adopted because of their high computational cost. In this work, we introduce the Random Error Sampling-based Neuroevolution (RESN), an evolutionary algorithm that uses the mean absolute error random sampling, a training-free approach to predict the expected performance of an artificial neural network, to optimize the architecture of a network. We empirically validate our proposal on three prediction problems, and compare our technique to training-based architecture optimization techniques and to neuroevolutionary approaches. Our findings show that we can achieve state-of-the-art error performance and that we reduce by half the time needed to perform the optimization.
Code (1)
Tasks
Hyperparameter OptimizationSimilar Papers 제목 키워드 기반
Reliable and Fast Recurrent Neural Network Architecture Optimization
This article introduces Random Error Sampling-based Neuroevolution (RESN), a novel automatic method to optimize recurrent neural network architectures. RESN combines an evolutionary algorithm with a training-free evaluat…
Adaptive Dataset Sampling by Deep Policy Gradient
Mini-batch SGD is a predominant optimization method in deep learning. Several works aim to improve naïve random dataset sampling, which appears typically in deep learning literature, with additional prior to allow faster…
image-classificationImage ClassificationBayesian Neural Architecture Search using A Training-Free Performance Metric
Recurrent neural networks (RNNs) are a powerful approach for time series prediction. However, their performance is strongly affected by their architecture and hyperparameter settings. The architecture optimization of RNN…
Bayesian OptimizationNeural Architecture SearchTime SeriesTime Series Analysis+1Stochastic optimization with arbitrary recurrent data sampling
For obtaining optimal first-order convergence guarantee for stochastic optimization, it is necessary to use a recurrent data sampling algorithm that samples every data point with sufficient frequency. Most commonly used …
Stochastic OptimizationLow-Cost Recurrent Neural Network Expected Performance Evaluation
Recurrent neural networks are a powerful tool, but they are very sensitive to their hyper-parameter configuration. Moreover, training properly a recurrent neural network is a tough task, therefore selecting an appropriat…