Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes
We consider Bayesian optimization of expensive-to-evaluate experiments that generate vector-valued outcomes over which a decision-maker (DM) has preferences. These preferences are encoded by a utility function that is not known in closed form but can be estimated by asking the DM to express preferences over pairs of outcome vectors. To address this problem, we develop Bayesian optimization with preference exploration, a novel framework that alternates between interactive real-time preference learning with the DM via pairwise comparisons between outcomes, and Bayesian optimization with a learned compositional model of DM utility and outcomes. Within this framework, we propose preference exploration strategies specifically designed for this task, and demonstrate their performance via extensive simulation studies.
Code (1)
Tasks
Bayesian OptimizationSimilar Papers 제목 키워드 기반
Bayesian Optimization with Preference Exploration by Monotonic Neural Network Ensemble
Many real-world black-box optimization problems have multiple conflicting objectives. Rather than attempting to approximate the entire set of Pareto-optimal solutions, interactive preference learning allows to focus the …
Bayesian OptimizationPreferential Bayesian Optimization with Crash Feedback
Bayesian optimization is a popular black-box optimization method for parameter learning in control and robotics. It typically requires an objective function that reflects the user's optimization goal. However, in practic…
BoTier: Multi-Objective Bayesian Optimization with Tiered Composite Objectives
Scientific optimization problems are usually concerned with balancing multiple competing objectives, which come as preferences over both the outcomes of an experiment (e.g. maximize the reaction yield) and the correspond…
Bayesian OptimizationExploration and preference satisfaction trade-off in reward-free learning
Biological agents have meaningful interactions with their environment despite the absence of immediate reward signals. In such instances, the agent can learn preferred modes of behaviour that lead to predictable states -…
OpenAI GymInverse Bayesian Optimization: Learning Human Acquisition Functions in an Exploration vs Exploitation Search Task
This paper introduces a probabilistic framework to estimate parameters of an acquisition function given observed human behavior that can be modeled as a collection of sample paths from a Bayesian optimization procedure. …
Bayesian Optimization