Hyper-parameter estimation method with particle swarm optimization
Particle swarm optimization (PSO) method cannot be directly used in the problem of hyper-parameter estimation since the mathematical formulation of the mapping from hyper-parameters to loss function or generalization accuracy is unclear. Bayesian optimization (BO) framework is capable of converting the optimization of the hyper-parameters into the optimization of an acquisition function. The acquisition function is non-convex and multi-peak. So the problem can be better solved by the PSO. The proposed method in this paper uses the particle swarm method to optimize the acquisition function in the BO framework to get better hyper-parameters. The performances of proposed method in both of the classification and regression models are evaluated and demonstrated. The results on several benchmark problems are improved.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian Optimizationparameter estimationregressionSimilar Papers 제목 키워드 기반
How to Prove the Optimized Values of Hyperparameters for Particle Swarm Optimization?
In recent years, several swarm intelligence optimization algorithms have been proposed to be applied for solving a variety of optimization problems. However, the values of several hyperparameters should be determined. Fo…
PositionQuadratically constrained quadratic programming for classification using particle swarms and applications
Particle swarm optimization is used in several combinatorial optimization problems. In this work, particle swarms are used to solve quadratic programming problems with quadratic constraints. The approach of particle swar…
Binary ClassificationClassificationCombinatorial OptimizationGeneral ClassificationOptimization of Convolutional Neural Network Using the Linearly Decreasing Weight Particle Swarm Optimization
Convolutional neural network (CNN) is one of the most frequently used deep learning techniques. Various forms of models have been proposed and im-proved for learning at CNN. When learning with CNN, it is necessary to det…
Hyperparameter OptimizationAdaptive Latent Factor Analysis via Generalized Momentum-Incorporated Particle Swarm Optimization
Stochastic gradient descent (SGD) algorithm is an effective learning strategy to build a latent factor analysis (LFA) model on a high-dimensional and incomplete (HDI) matrix. A particle swarm optimization (PSO) algorithm…
A Particle Swarm Optimization hyper-heuristic for the Dynamic Vehicle Routing Problem
This paper presents a method for choosing a Particle Swarm Optimization based optimizer for the Dynamic Vehicle Routing Problem on the basis of the initially available data of a given problem instance. The optimization a…