Predictive Representations for Skill Transfer in Reinforcement Learning

2026-04-08 · Ruben Vereecken, Luke Dickens, Alessandra Russo arxiv

A key challenge in scaling up Reinforcement Learning is generalizing learned behaviour. Without the ability to carry forward acquired knowledge an agent is doomed to learn each task from scratch. In this paper we develop a new formalism for transfer by virtue of state abstraction. Based on task-independent, compact observations (outcomes) of the environment, we introduce Outcome-Predictive State Representations (OPSRs), agent-centered and task-independent abstractions that are made up of predictions of outcomes. We show formally and empirically that they have the potential for optimal but limited transfer, then overcome this trade-off by introducing OPSR-based skills, i.e. abstract actions (based on options) that can be reused between tasks as a result of state abstraction. In a series of empirical studies, we learn OPSR-based skills from demonstrations and show how they speed up learning considerably in entirely new and unseen tasks without any pre-processing. We believe that the framework introduced in this work is a promising step towards transfer in RL in general, and towards transfer through combining state and action abstraction specifically.

📄 PDF Abstract BibTeX arXiv:2604.07016

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Simulator Predictive Control: Using Learned Task Representations and MPC for Zero-Shot Generalization and Sequencing

2018-10-04 · Zhanpeng He, Ryan Julian, Eric Heiden, Hejia Zhang 외

Simulation-to-real transfer is an important strategy for making reinforcement learning practical with real robots. Successful sim-to-real transfer systems have difficulty producing policies which generalize across tasks,…

Model Predictive Control Reinforcement Learning Zero-shot Generalization

PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale

2022-10-15 · Kuang-Huei Lee, Ted Xiao, Adrian Li, Paul Wohlhart 외

The predictive information, the mutual information between the past and future, has been shown to be a useful representation learning auxiliary loss for training reinforcement learning agents, as the ability to model wha…

reinforcement-learning Reinforcement Learning (RL)Representation Learning Robot Manipulation

ACNMP: Skill Transfer and Task Extrapolation through Learning from Demonstration and Reinforcement Learning via Representation Sharing

2020-03-25 · M. Tuluhan Akbulut, Erhan Oztop, M. Yunus Seker, Honghu Xue 외

To equip robots with dexterous skills, an effective approach is to first transfer the desired skill via Learning from Demonstration (LfD), then let the robot improve it by self-exploration via Reinforcement Learning (RL)…

Reinforcement Learning Reinforcement Learning (RL)

Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning

2021-10-25 · NeurIPS 2021 12 · Jinxin Liu, Hao Shen, Donglin Wang, Yachen Kang 외

Unsupervised reinforcement learning aims to acquire skills without prior goal representations, where an agent automatically explores an open-ended environment to represent goals and learn the goal-conditioned policy. How…

Domain Adaptation reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+2

Robotic self-representation improves manipulation skills and transfer learning

2020-11-13 · Phuong D. H. Nguyen, Manfred Eppe, Stefan Wermter

Cognitive science suggests that the self-representation is critical for learning and problem-solving. However, there is a lack of computational methods that relate this claim to cognitively plausible robots and reinforce…

reinforcement-learning Reinforcement Learning (RL)Transfer Learning