Zero-Shot Offline Imitation Learning via Optimal Transport
Zero-shot imitation learning algorithms hold the promise of reproducing unseen behavior from as little as a single demonstration at test time. Existing practical approaches view the expert demonstration as a sequence of goals, enabling imitation with a high-level goal selector, and a low-level goal-conditioned policy. However, this framework can suffer from myopic behavior: the agent's immediate actions towards achieving individual goals may undermine long-term objectives. We introduce a novel method that mitigates this issue by directly optimizing the occupancy matching objective that is intrinsic to imitation learning. We propose to lift a goal-conditioned value function to a distance between occupancies, which are in turn approximated via a learned world model. The resulting method can learn from offline, suboptimal data, and is capable of non-myopic, zero-shot imitation, as we demonstrate in complex, continuous benchmarks.
Code (1)
Tasks
Imitation LearningSimilar Papers 제목 키워드 기반
Improving Zero-Shot Offline RL via Behavioral Task Sampling
Offline zero-shot reinforcement learning (RL) aims to learn agents that optimize unseen reward functions without additional environment interaction. The standard approach to this problem trains task-conditioned policies …
Zero-shot GeneralizationReinforcement LearningOffline RLZero-Shot Recognition via Optimal Transport
We propose an optimal transport (OT) framework for generalized zero-shot learning (GZSL), seeking to distinguish samples for both seen and unseen classes, with the assist of auxiliary attributes. The discrepancy between …
AttributeGeneralized Zero-Shot LearningZero-Shot LearningAlign Your Intents: Offline Imitation Learning via Optimal Transport
Offline Reinforcement Learning (RL) addresses the problem of sequential decision-making by learning optimal policy through pre-collected data, without interacting with the environment. As yet, it has remained somewhat im…
D4RLDecision MakingImitation LearningOffline RL+2ZegOT: Zero-shot Segmentation Through Optimal Transport of Text Prompts
Recent success of large-scale Contrastive Language-Image Pre-training (CLIP) has led to great promise in zero-shot semantic segmentation by transferring image-text aligned knowledge to pixel-level classification. However…
SegmentationSemantic SegmentationZero Shot SegmentationZero-Shot Semantic SegmentationTransductive Universal Transport for Zero-Shot Action Recognition
This work addresses the problem of recognizing action categories in videos for which no training examples are available. The current state-of-the-art enables such a zero-shot recognition by learning universal mappings fr…
Action RecognitionObjectPositionTemporal Localization+3