Lipschitz Lifelong Reinforcement Learning

2020-01-15 · Erwan Lecarpentier, David Abel, Kavosh Asadi, Yuu Jinnai, Emmanuel Rachelson, Michael L. Littman

We consider the problem of knowledge transfer when an agent is facing a series of Reinforcement Learning (RL) tasks. We introduce a novel metric between Markov Decision Processes (MDPs) and establish that close MDPs have close optimal value functions. Formally, the optimal value functions are Lipschitz continuous with respect to the tasks space. These theoretical results lead us to a value-transfer method for Lifelong RL, which we use to build a PAC-MDP algorithm with improved convergence rate. Further, we show the method to experience no negative transfer with high probability. We illustrate the benefits of the method in Lifelong RL experiments.

📄 PDF Abstract BibTeX arXiv:2001.05411

Code (1)

SuReLI/llrl 공식 구현

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Some Insights into Lifelong Reinforcement Learning Systems

2020-01-27 · Changjian Li

A lifelong reinforcement learning system is a learning system that has the ability to learn through trail-and-error interaction with the environment over its lifetime. In this paper, I give some arguments to show that th…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Bayesian Exploration for Lifelong Reinforcement Learning

2021-09-29 · Haotian Fu, Shangqun Yu, Michael Littman, George Konidaris

A central question in reinforcement learning (RL) is how to leverage prior knowledge to accelerate learning in new tasks. We propose a Bayesian exploration method for lifelong reinforcement learning (BLRL) that aims to …

Lifelong learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Safe Policy Search for Lifelong Reinforcement Learning with Sublinear Regret

2015-05-21 · Haitham Bou Ammar, Rasul Tutunov, Eric Eaton

Lifelong reinforcement learning provides a promising framework for developing versatile agents that can accumulate knowledge over a lifetime of experience and rapidly learn new tasks by building upon prior knowledge. How…

Lifelong learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Sharing Lifelong Reinforcement Learning Knowledge via Modulating Masks

2023-05-18 · Saptarshi Nath, Christos Peridis, Eseoghene Ben-Iwhiwhu, Xinran Liu 외

Lifelong learning agents aim to learn multiple tasks sequentially over a lifetime. This involves the ability to exploit previous knowledge when learning new tasks and to avoid forgetting. Modulating masks, a specific typ…

Lifelong learning reinforcement-learning Reinforcement Learning

Model-based Lifelong Reinforcement Learning with Bayesian Exploration

2022-10-20 · Haotian Fu, Shangqun Yu, Michael Littman, George Konidaris

We propose a model-based lifelong reinforcement-learning approach that estimates a hierarchical Bayesian posterior distilling the common structure shared across different tasks. The learned posterior combined with a samp…

model reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)