Model-based Lifelong Reinforcement Learning with Bayesian Exploration
We propose a model-based lifelong reinforcement-learning approach that estimates a hierarchical Bayesian posterior distilling the common structure shared across different tasks. The learned posterior combined with a sample-based Bayesian exploration procedure increases the sample efficiency of learning across a family of related tasks. We first derive an analysis of the relationship between the sample complexity and the initialization quality of the posterior in the finite MDP setting. We next scale the approach to continuous-state domains by introducing a Variational Bayesian Lifelong Reinforcement Learning algorithm that can be combined with recent model-based deep RL methods, and that exhibits backward transfer. Experimental results on several challenging domains show that our algorithms achieve both better forward and backward transfer performance than state-of-the-art lifelong RL methods.
Code (2)
Tasks
modelreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Bayesian Exploration for Lifelong Reinforcement Learning
A central question in reinforcement learning (RL) is how to leverage prior knowledge to accelerate learning in new tasks. We propose a Bayesian exploration method for lifelong reinforcement learning (BLRL) that aims to …
Lifelong learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learning
In lifelong learning, an agent learns throughout its entire life without resets, in a constantly changing environment, as we humans do. Consequently, lifelong learning comes with a plethora of research problems such as c…
Lifelong learningPolicy Gradient MethodsQ-Learningreinforcement-learning+1ELEMENT: Episodic and Lifelong Exploration via Maximum Entropy
This paper proposes \emph{Episodic and Lifelong Exploration via Maximum ENTropy} (ELEMENT), a novel, multiscale, intrinsically motivated reinforcement learning (RL) framework that is able to explore environments without …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Lifelong Incremental Reinforcement Learning with Online Bayesian Inference
A central capability of a long-lived reinforcement learning (RL) agent is to incrementally adapt its behavior as its environment changes, and to incrementally build upon previous experiences to facilitate future learning…
Bayesian InferenceClusteringLifelong learningreinforcement-learning+2State Abstractions for Lifelong Reinforcement Learning
In lifelong reinforcement learning, agents must effectively transfer knowledge across tasks while simultaneously addressing exploration, credit assignment, and generalization. State abstraction can help overcome the…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)