홈 › Papers

Successor Features for Transfer in Reinforcement Learning

2016-06-16 · NeurIPS 2017 12 · André Barreto, Will Dabney, Rémi Munos, Jonathan J. Hunt, Tom Schaul, Hado van Hasselt, David Silver

Transfer in reinforcement learning refers to the notion that generalization should occur not only within a task but also across tasks. We propose a transfer framework for the scenario where the reward function changes between tasks but the environment's dynamics remain the same. Our approach rests on two key ideas: "successor features", a value function representation that decouples the dynamics of the environment from the rewards, and "generalized policy improvement", a generalization of dynamic programming's policy improvement operation that considers a set of policies rather than a single one. Put together, the two ideas lead to an approach that integrates seamlessly within the reinforcement learning framework and allows the free exchange of information across tasks. The proposed method also provides performance guarantees for the transferred policy even before any learning has taken place. We derive two theorems that set our approach in firm theoretical ground and present experiments that show that it successfully promotes transfer in practice, significantly outperforming alternative methods in a sequence of navigation tasks and in the control of a simulated robotic arm.

📄 PDF Abstract BibTeX arXiv:1606.05312

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning

2017-07-31 · Lucas Lehnert, Stefanie Tellex, Michael L. Littman

One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks. Successor Features approach this problem by …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A New Representation of Successor Features for Transfer across Dissimilar Environments

2021-07-18 · Majid Abdolshah, Hung Le, Thommen Karimpanal George, Sunil Gupta 외

Transfer in reinforcement learning is usually achieved through generalisation across tasks. Whilst many studies have investigated transferring knowledge when the reward function changes, they have assumed that the dynami…

Gaussian Processes Reinforcement Learning (RL)

Successor Feature Representations

2021-10-29 · Chris Reinke, Xavier Alameda-Pineda

Transfer in Reinforcement Learning aims to improve learning performance on target tasks using knowledge from experienced source tasks. Successor Representations (SR) and their extension Successor Features (SF) are promin…

Transfer Learning

Universal Successor Features for Transfer Reinforcement Learning

2020-01-05 · ICLR 2019 5 · Chen Ma, Dylan R. Ashley, Junfeng Wen, Yoshua Bengio

Transfer in Reinforcement Learning (RL) refers to the idea of applying knowledge gained from previous tasks to solving related tasks. Learning a universal value function (Schaul et al., 2015), which generalizes over goal…

MuJoCo reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Transfer with Model Features in Reinforcement Learning

2018-07-04 · Lucas Lehnert, Michael L. Littman

A key question in Reinforcement Learning is which representation an agent can learn to efficiently reuse knowledge between different tasks. Recently the Successor Representation was shown to have empirical benefits for t…

model reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)