Rollout Algorithms and Approximate Dynamic Programming for Bayesian Optimization and Sequential Estimation
We provide a unifying approximate dynamic programming framework that applies to a broad variety of problems involving sequential estimation. We consider first the construction of surrogate cost functions for the purposes of optimization, and we focus on the special case of Bayesian optimization, using the rollout algorithm and some of its variations. We then discuss the more general case of sequential estimation of a random vector using optimal measurement selection, and its application to problems of stochastic and adaptive control. We distinguish between adaptive control of deterministic and stochastic systems: the former are better suited for the use of rollout, while the latter are well suited for the use of rollout with certainty equivalence approximations. As an example of the deterministic case, we discuss sequential decoding problems, and a rollout algorithm for the approximate solution of the Wordle and Mastermind puzzles, recently developed in the paper [BBB22].
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationSimilar Papers 제목 키워드 기반
Bayesian Optimization with a Finite Budget: An Approximate Dynamic Programming Approach
We consider the problem of optimizing an expensive objective function when a finite budget of total evaluations is prescribed. In that context, the optimal solution strategy for Bayesian optimization can be formulated as…
Bayesian OptimizationThe Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation
Function approximation is widely used in reinforcement learning to handle the computational difficulties associated with very large state spaces. However, function approximation introduces errors which may lead to instab…
Multiagent Rollout and Policy Iteration for POMDP with Application to Multi-Robot Repair Problems
In this paper we consider infinite horizon discounted dynamic programming problems with finite state and control spaces, partial state observations, and a multiagent structure. We discuss and compare algorithms that simu…
Multiagent Rollout Algorithms and Reinforcement Learning
We consider finite and infinite horizon dynamic programming problems, where the control at each stage consists of several distinct decisions, each one made by one of several agents. We introduce an approach, whereby at e…
Computational Efficiencyreinforcement-learningReinforcement LearningReinforcement Learning (RL)Inferential Induction: A Novel Framework for Bayesian Reinforcement Learning
Bayesian reinforcement learning (BRL) offers a decision-theoretic solution for reinforcement learning. While "model-based" BRL algorithms have focused either on maintaining a posterior distribution on models or value fun…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)