Mixed-Variable Bayesian Optimization
The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete domains have been widely studied. However, few methods exist for mixed-variable domains and none of them can handle discrete constraints that arise in many real-world applications. In this paper, we introduce MiVaBo, a novel BO algorithm for the efficient optimization of mixed-variable functions combining a linear surrogate model based on expressive feature representations with Thompson sampling. We propose an effective method to optimize its acquisition function, a challenging problem for mixed-variable domains, making MiVaBo the first BO method that can handle complex constraints over the discrete variables. Moreover, we provide the first convergence analysis of a mixed-variable BO algorithm. Finally, we show that MiVaBo is significantly more sample efficient than state-of-the-art mixed-variable BO algorithms on several hyperparameter tuning tasks, including the tuning of deep generative models.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationThompson SamplingSimilar Papers 제목 키워드 기반
A comparison of mixed-variables Bayesian optimization approaches
Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically c…
Bayesian OptimizationGaussian ProcessesHybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization
This paper presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed n…
Bayesian OptimizationGaussian ProcessesModel SelectionPositionBayesian Optimization For Multi-Objective Mixed-Variable Problems
Optimizing multiple, non-preferential objectives for mixed-variable, expensive black-box problems is important in many areas of engineering and science. The expensive, noisy, black-box nature of these problems makes them…
Bayesian OptimizationHybrid Reinforcement Learning Framework for Mixed-Variable Problems
Optimization problems characterized by both discrete and continuous variables are common across various disciplines, presenting unique challenges due to their complex solution landscapes and the difficulty of navigating …
Bayesian Optimizationreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables
Although Bayesian Optimization (BO) has been employed for accelerating materials design in computational materials engineering, existing works are restricted to problems with quantitative variables. However, real designs…
Bayesian Optimization