Training Feedforward Neural Networks with Bayesian Hyper-Heuristics
The process of training feedforward neural networks (FFNNs) can benefit from an automated process where the best heuristic to train the network is sought out automatically by means of a high-level probabilistic-based heuristic. This research introduces a novel population-based Bayesian hyper-heuristic (BHH) that is used to train feedforward neural networks (FFNNs). The performance of the BHH is compared to that of ten popular low-level heuristics, each with different search behaviours. The chosen heuristic pool consists of classic gradient-based heuristics as well as meta-heuristics (MHs). The empirical process is executed on fourteen datasets consisting of classification and regression problems with varying characteristics. The BHH is shown to be able to train FFNNs well and provide an automated method for finding the best heuristic to train the FFNNs at various stages of the training process.
Code (1)
Similar Papers 제목 키워드 기반
Towards Automatic Bayesian Optimization: A first step involving acquisition functions
Bayesian Optimization is the state of the art technique for the optimization of black boxes, i.e., functions where we do not have access to their analytical expression nor its gradients, they are expensive to evaluate an…
Bayesian OptimizationBIG-bench Machine LearningMetaheuristics optimized feedforward neural networks for efficient stock price prediction
The prediction of stock prices is an important task in economics, investment and making financial decisions. This has, for decades, spurred the interest of many researchers to make focused contributions to the design of …
Decision MakingStock Price PredictionTime SeriesTime Series AnalysisLearning Domain-Independent Planning Heuristics with Hypergraph Networks
We present the first approach capable of learning domain-independent planning heuristics entirely from scratch. The heuristics we learn map the hypergraph representation of the delete-relaxation of the planning problem a…
Bayesian Structure Scores for Probabilistic Circuits
Probabilistic circuits (PCs) are a prominent representation of probability distributions with tractable inference. While parameter learning in PCs is rigorously studied, structure learning is often more based on heuristi…
How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS
Weight sharing promises to make neural architecture search (NAS) tractable even on commodity hardware. Existing methods in this space rely on a diverse set of heuristics to design and train the shared-weight backbone net…
Neural Architecture Search