Contextual Bayesian optimization with binary outputs
Bayesian optimization (BO) is an efficient method to optimize expensive black-box functions. It has been generalized to scenarios where objective function evaluations return stochastic binary feedback, such as success/failure in a given test, or preference between different parameter settings. In many real-world situations, the objective function can be evaluated in controlled 'contexts' or 'environments' that directly influence the observations. For example, one could directly alter the 'difficulty' of the test that is used to evaluate a system's performance. With binary feedback, the context determines the information obtained from each observation. For example, if the test is too easy/hard, the system will always succeed/fail, yielding uninformative binary outputs. Here we combine ideas from Bayesian active learning and optimization to efficiently choose the best context and optimization parameter on each iteration. We demonstrate the performance of our algorithm and illustrate how it can be used to tackle a concrete application in visual psychophysics: efficiently improving patients' vision via corrective lenses, using psychophysics measurements.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningBayesian OptimizationSimilar Papers 제목 키워드 기반
Robust Bayesian Classification Using an Optimistic Score Ratio
We build a Bayesian contextual classification model using an optimistic score ratio for robust binary classification when there is limited information on the class-conditional, or contextual, distribution. The optimistic…
Binary ClassificationClassificationGeneral ClassificationOptimizing High-Dimensional Physics Simulations via Composite Bayesian Optimization
Physical simulation-based optimization is a common task in science and engineering. Many such simulations produce image- or tensor-based outputs where the desired objective is a function of those outputs, and optimizatio…
Bayesian OptimizationVocal Bursts Intensity PredictionCollaborative Contextual Bayesian Optimization
Discovering optimal designs through sequential data collection is essential in many real-world applications. While Bayesian Optimization (BO) has achieved remarkable success in this setting, growing attention has recentl…
On the Generalization of the C-Bound to Structured Output Ensemble Methods
This paper generalizes an important result from the PAC-Bayesian literature for binary classification to the case of ensemble methods for structured outputs. We prove a generic version of the \Cbound, an upper bound over…
Binary ClassificationGeneral ClassificationHigh-Dimensional Contextual Policy Search with Unknown Context Rewards using Bayesian Optimization
Contextual policies are used in many settings to customize system parameters and actions to the specifics of a particular setting. In some real-world settings, such as randomized controlled trials or A/B tests, it may no…
Bayesian Optimization