Multitask Adaptation by Retrospective Exploration with Learned World Models
Model-based reinforcement learning (MBRL) allows solving complex tasks in a sample-efficient manner. However, no information is reused between the tasks. In this work, we propose a meta-learned addressing model called RAMa that provides training samples for the MBRL agent taken from continuously growing task-agnostic storage. The model is trained to maximize the expected agent's performance by selecting promising trajectories solving prior tasks from the storage. We show that such retrospective exploration can accelerate the learning process of the MBRL agent by better informing learned dynamics and prompting agent with exploratory trajectories. We test the performance of our approach on several domains from the DeepMind control suite, from Metaworld multitask benchmark, and from our bespoke environment implemented with a robotic NVIDIA Isaac simulator to test the ability of the model to act in a photorealistic, ray-traced environment.
Code (0)
등록된 구현이 없습니다.
Tasks
Model-based Reinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RetroAgent: From Solving to Evolving via Retrospective Dual Intrinsic Feedback
Standard reinforcement learning (RL) for large language model (LLM) agents primarily optimizes extrinsic task rewards, often favoring isolated task completion over continual adaptation. This paradigm can cause premature …
Reinforcement LearningTest-time AdaptationPlanning to Explore via Self-Supervised World Models
Reinforcement learning allows solving complex tasks, however, the learning tends to be task-specific and the sample efficiency remains a challenge. We present Plan2Explore, a self-supervised reinforcement learning agent …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Bayesian Multitask Learning with Latent Hierarchies
We learn multiple hypotheses for related tasks under a latent hierarchical relationship between tasks. We exploit the intuition that for domain adaptation, we wish to share classifier structure, but for multitask learnin…
Domain AdaptationProvable Benefit of Multitask Representation Learning in Reinforcement Learning
As representation learning becomes a powerful technique to reduce sample complexity in reinforcement learning (RL) in practice, theoretical understanding of its advantage is still limited. In this paper, we theoretically…
Offline RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)+1Vision Transformer Adapters for Generalizable Multitask Learning
We introduce the first multitasking vision transformer adapters that learn generalizable task affinities which can be applied to novel tasks and domains. Integrated into an off-the-shelf vision transformer backbone, our …
Domain AdaptationUnsupervised Domain Adaptation