Dual Control for Approximate Bayesian Reinforcement Learning
Control of non-episodic, finite-horizon dynamical systems with uncertain dynamics poses a tough and elementary case of the exploration-exploitation trade-off. Bayesian reinforcement learning, reasoning about the effect of actions and future observations, offers a principled solution, but is intractable. We review, then extend an old approximate approach from control theory---where the problem is known as dual control---in the context of modern regression methods, specifically generalized linear regression. Experiments on simulated systems show that this framework offers a useful approximation to the intractable aspects of Bayesian RL, producing structured exploration strategies that differ from standard RL approaches. We provide simple examples for the use of this framework in (approximate) Gaussian process regression and feedforward neural networks for the control of exploration.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Revised Progressive-Hedging-Algorithm Based Two-layer Solution Scheme for Bayesian Reinforcement Learning
Stochastic control with both inherent random system noise and lack of knowledge on system parameters constitutes the core and fundamental topic in reinforcement learning (RL), especially under non-episodic situations whe…
Reinforcement LearningReinforcement Learning (RL)Thompson SamplingPAC-Bayesian Soft Actor-Critic Learning
Actor-critic algorithms address the dual goals of reinforcement learning (RL), policy evaluation and improvement via two separate function approximators. The practicality of this approach comes at the expense of training…
Reinforcement Learning (RL)Scalable Bayesian Inverse Reinforcement Learning
Bayesian inference over the reward presents an ideal solution to the ill-posed nature of the inverse reinforcement learning problem. Unfortunately current methods generally do not scale well beyond the small tabular sett…
Bayesian InferenceImitation Learningreinforcement-learningReinforcement Learning+1Approximating Pareto Frontier through Bayesian-optimization-directed Robust Multi-objective Reinforcement Learning
Many real-word decision or control problems involve multiple conflicting objectives and uncertainties, which requires learned policies are not only Pareto optimal but also robust. In this paper, we proposed a novel algor…
Bayesian OptimizationDiversityMulti-Objective Reinforcement Learningreinforcement-learning+2Residual Deep Reinforcement Learning for Inverter-based Volt-Var Control
A residual deep reinforcement learning (RDRL) approach is proposed by integrating DRL with model-based optimization for inverter-based volt-var control in active distribution networks when the accurate power flow model i…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning