BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
Bayesian optimization provides sample-efficient global optimization for a broad range of applications, including automatic machine learning, engineering, physics, and experimental design. We introduce BoTorch, a modern programming framework for Bayesian optimization that combines Monte-Carlo (MC) acquisition functions, a novel sample average approximation optimization approach, auto-differentiation, and variance reduction techniques. BoTorch's modular design facilitates flexible specification and optimization of probabilistic models written in PyTorch, simplifying implementation of new acquisition functions. Our approach is backed by novel theoretical convergence results and made practical by a distinctive algorithmic foundation that leverages fast predictive distributions, hardware acceleration, and deterministic optimization. We also propose a novel "one-shot" formulation of the Knowledge Gradient, enabled by a combination of our theoretical and software contributions. In experiments, we demonstrate the improved sample efficiency of BoTorch relative to other popular libraries.
Code (2)
Tasks
Bayesian OptimisationBayesian OptimizationExperimental Designglobal-optimizationSimilar Papers 제목 키워드 기반
Bayesian Hyperparameter Optimization with BoTorch, GPyTorch and Ax
Deep learning models are full of hyperparameters, which are set manually before the learning process can start. To find the best configuration for these hyperparameters in such a high dimensional space, with time-consumi…
Bayesian OptimizationGaussian ProcessesGraph ClassificationHyperparameter OptimizationBenchmarking Generative AI Against Bayesian Optimization for Constrained Multi-Objective Inverse Design
This paper investigates the performance of Large Language Models (LLMs) as generative optimizers for solving constrained multi-objective regression tasks, specifically within the challenging domain of inverse design (pro…
parameter-efficient fine-tuningUnbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs
In this paper we propose an efficient stochastic optimization algorithm to search for Bayesian experimental designs such that the expected information gain is maximized. The gradient of the expected information gain with…
Experimental DesignStochastic OptimizationBayesian optimization for backpropagation in Monte-Carlo tree search
In large domains, Monte-Carlo tree search (MCTS) is required to estimate the values of the states as efficiently and accurately as possible. However, the standard update rule in backpropagation assumes a stationary distr…
Bayesian OptimizationAccelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need
We leverage multilevel Monte Carlo (MLMC) to improve the performance of multi-step look-ahead Bayesian optimization (BO) methods that involve nested expectations and maximizations. Often these expectations must be comput…
AllBayesian Optimization