Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following
Effective task representations should facilitate compositionality, such that after learning a variety of basic tasks, an agent can perform compound tasks consisting of multiple steps simply by composing the representations of the constituent steps together. While this is conceptually simple and appealing, it is not clear how to automatically learn representations that enable this sort of compositionality. We show that learning to associate the representations of current and future states with a temporal alignment loss can improve compositional generalization, even in the absence of any explicit subtask planning or reinforcement learning. We evaluate our approach across diverse robotic manipulation tasks as well as in simulation, showing substantial improvements for tasks specified with either language or goal images.
Code (0)
등록된 구현이 없습니다.
Tasks
Instruction FollowingSimilar Papers 제목 키워드 기반
Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning
One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks. Successor Features approach this problem by …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Successor Features Combine Elements of Model-Free and Model-based Reinforcement Learning
A key question in reinforcement learning is how an intelligent agent can generalize knowledge across different inputs. By generalizing across different inputs, information learned for one input can be immediately reused …
Decision MakingmodelModel-based Reinforcement Learningreinforcement-learning+2Combining Behaviors with the Successor Features Keyboard
The Option Keyboard (OK) was recently proposed as a method for transferring behavioral knowledge across tasks. OK transfers knowledge by adaptively combining subsets of known behaviors using Successor Features (SFs) and …
Transfer LearningTemporally Extended Successor Representations
We present a temporally extended variation of the successor representation, which we term t-SR. t-SR captures the expected state transition dynamics of temporally extended actions by constructing successor representation…
Switching Successor Measures for Hierarchical Zero-shot Reinforcement Learning
Hierarchical reinforcement learning can improve generalization by decomposing long-horizon decision-making into simpler subproblems. However, existing approaches often rely on restrictive design choices, such as fixed te…
Hierarchical Reinforcement Learning