Directed particle swarm optimization with Gaussian-process-based function forecasting
Particle swarm optimization (PSO) is an iterative search method that moves a set of candidate solution around a search-space towards the best known global and local solutions with randomized step lengths. PSO frequently accelerates optimization in practical applications, where gradients are not available and function evaluations expensive. Yet the traditional PSO algorithm ignores the potential knowledge that could have been gained of the objective function from the observations by individual particles. Hence, we draw upon concepts from Bayesian optimization and introduce a stochastic surrogate model of the objective function. That is, we fit a Gaussian process to past evaluations of the objective function, forecast its shape and then adapt the particle movements based on it. Our computational experiments demonstrate that baseline implementations of PSO (i.e., SPSO2011) are outperformed. Furthermore, compared to, state-of-art surrogate-assisted evolutionary algorithms, we achieve substantial performance improvements on several popular benchmark functions. Overall, we find that our algorithm attains desirable properties for exploratory and exploitative behavior.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationEvolutionary AlgorithmsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Replica Exchange using q-Gaussian Swarm Quantum Particle Intelligence Method
We present a newly developed Replica Exchange algorithm using q -Gaussian Swarm Quantum Particle Optimization (REX@q-GSQPO) method for solving the problem of finding the global optimum. The basis of the algorithm is to r…
DiversityQ-Gaussian Swarm Quantum Particle Intelligence on Predicting Global Minimum of Potential Energy Function
We present a newly developed -Gaussian Swarm Quantum-like Particle Optimization (q-GSQPO) algorithm to determine the global minimum of the potential energy function. Swarm Quantum-like Particle Optimization (SQPO) algori…
AMPSO: Artificial Multi-Swarm Particle Swarm Optimization
In this paper we propose a novel artificial multi-swarm PSO which consists of an exploration swarm, an artificial exploitation swarm and an artificial convergence swarm. The exploration swarm is a set of equal-sized sub-…
DiversityA Particle-based Sparse Gaussian Process Optimizer
Task learning in neural networks typically requires finding a globally optimal minimizer to a loss function objective. Conventional designs of swarm based optimization methods apply a fixed update rule, with possibly an …
image-classificationImage ClassificationHeterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems
We propose Heterogeneous Swarms, an algorithm to design multi-LLM systems by jointly optimizing model roles and weights. We represent multi-LLM systems as directed acyclic graphs (DAGs) of LLMs with topological message p…