Generalizing Successor Features to continuous domains for Multi-task Learning

2021-09-29 · Melissa Mozifian, Dieter Fox, David Meger, Fabio Ramos, Animesh Garg

The deep reinforcement learning (RL) framework has shown great promise to tackle sequential decision-making problems, where the agent learns to behave optimally through interactions with the environment and receiving rewards. The ability of an RL agent to learn different reward functions concurrently has many benefits, such as the decomposition of task rewards and skill reuse. One obstacle for achieving this, is the amount of data required as well as the capacity of the model for solving multiple tasks. In this paper, we consider the problem of continuous control for various robot manipulation tasks with an explicit representation that promotes skill reuse while learning multiple tasks, related through the reward function. Our approach relies on two key concepts: successor features (SF), a value function representation that decouples the dynamics of the environment from the rewards, and an actor-critic framework that incorporates the learned SF representations. We propose a practical implementation of successor features in continuous action spaces. We first show how to learn a decomposable representation required by SF. Our proposed methods, is able to learn decoupled state and reward features representations. We study this approach on a non-trivial continuous control problems with compositional structure built into the reward functions of various tasks.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Decision MakingDeep Reinforcement LearningMulti-Task Learning Reinforcement Learning (RL)Robot ManipulationSequential Decision Making

Full-Gradient Successor Feature Representations

2026-04-01 · Ritish Shrirao, Aditya Priyadarshi, Raghuram Bharadwaj Diddigi arxiv

Successor Features (SF) combined with Generalized Policy Improvement (GPI) provide a robust framework for transfer learning in Reinforcement Learning (RL) by decoupling environment dynamics from reward functions. However…

Reinforcement Learning Transfer Learning

Successor Features Combine Elements of Model-Free and Model-based Reinforcement Learning

2019-01-31 · Lucas Lehnert, Michael L. Littman

A key question in reinforcement learning is how an intelligent agent can generalize knowledge across different inputs. By generalizing across different inputs, information learned for one input can be immediately reused …

Decision Making model Model-based Reinforcement Learning reinforcement-learning+2

Fast Task Inference with Variational Intrinsic Successor Features

2019-06-12 · ICLR 2020 1 · Steven Hansen, Will Dabney, Andre Barreto, Tom Van de Wiele 외

It has been established that diverse behaviors spanning the controllable subspace of an Markov decision process can be trained by rewarding a policy for being distinguishable from other policies \citep{gregor2016variatio…

Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning

2017-07-31 · Lucas Lehnert, Stefanie Tellex, Michael L. Littman

One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks. Successor Features approach this problem by …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Successor Feature Representations

2021-10-29 · Chris Reinke, Xavier Alameda-Pineda

Transfer in Reinforcement Learning aims to improve learning performance on target tasks using knowledge from experienced source tasks. Successor Representations (SR) and their extension Successor Features (SF) are promin…

Transfer Learning