Batch Multi-Fidelity Bayesian Optimization with Deep Auto-Regressive Networks
Bayesian optimization (BO) is a powerful approach for optimizing black-box, expensive-to-evaluate functions. To enable a flexible trade-off between the cost and accuracy, many applications allow the function to be evaluated at different fidelities. In order to reduce the optimization cost while maximizing the benefit-cost ratio, in this paper, we propose Batch Multi-fidelity Bayesian Optimization with Deep Auto-Regressive Networks (BMBO-DARN). We use a set of Bayesian neural networks to construct a fully auto-regressive model, which is expressive enough to capture strong yet complex relationships across all the fidelities, so as to improve the surrogate learning and optimization performance. Furthermore, to enhance the quality and diversity of queries, we develop a simple yet efficient batch querying method, without any combinatorial search over the fidelities. We propose a batch acquisition function based on Max-value Entropy Search (MES) principle, which penalizes highly correlated queries and encourages diversity. We use posterior samples and moment matching to fulfill efficient computation of the acquisition function and conduct alternating optimization over every fidelity-input pair, which guarantees an improvement at each step. We demonstrate the advantage of our approach on four real-world hyperparameter optimization applications.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationDiversityHyperparameter OptimizationSimilar Papers 제목 키워드 기반
Combining Multi-Fidelity Modelling and Asynchronous Batch Bayesian Optimization
Bayesian Optimization is a useful tool for experiment design. Unfortunately, the classical, sequential setting of Bayesian Optimization does not translate well into laboratory experiments, for instance battery design, wh…
Bayesian OptimizationBatch Bayesian Optimization via Multi-objective Acquisition Ensemble for Automated Analog Circuit Design
Bayesian optimization methods are promising for the optimization of black-box functions that are expensive to evaluate. In this paper, a novel batch Bayesian optimization approach is proposed. The parallelization is…
Bayesian OptimizationMulti-fidelity batch Bayesian optimization for bioprocess development across scales
Bioprocesses are central to modern biotechnology, enabling sustainable production of pharmaceuticals, specialty chemicals, cosmetics, and food. However, developing high-performing processes remains costly and complex, re…
Gaussian ProcessesTransfer LearningaphBO-2GP-3B: A budgeted asynchronous parallel multi-acquisition functions for constrained Bayesian optimization on high-performing computing architecture
High-fidelity complex engineering simulations are highly predictive, but also computationally expensive and often require substantial computational efforts. The mitigation of computational burden is usually enabled throu…
Bayesian OptimizationGaussian ProcessesAdaptive Model Refinement with Batch Bayesian Sampling for Optimization of Bio-inspired Flow Tailoring
This paper presents an advancement to an approach for model-independent surrogate-based optimization with adaptive batch sampling, known as Adaptive Model Refinement (AMR). While the original AMR method provides unique d…
Bayesian Optimization