Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance
Much like humans benefit from guidance while learning, reinforcement learning algorithms may benefit from additional supervision beyond rewards. Leveraging additional information during training to learn better representations and behaviors has been the focus of asymmetric reinforcement learning. This learning paradigm has proven effective under partial observability when additional state information is available, but also under full observability when more refined state information is available. Focusing on model-based reinforcement learning, we study the effect of asymmetric learning on observation representations and on privileged information representations. First, we identify a limitation in the privileged information representations learned by an asymmetric model-based algorithm known as the Informed Dreamer. Then, we propose a novel asymmetric representation learning objective using latent guidance, resulting in a new algorithm called the Reinformed Dreamer. Experiments across several benchmarks show a more consistent improvement over Dreamer than previous asymmetric approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningReinforcement LearningSimilar Papers 제목 키워드 기반
PIGDreamer: Privileged Information Guided World Models for Safe Partially Observable Reinforcement Learning
Partial observability presents a significant challenge for Safe Reinforcement Learning (Safe RL), as it impedes the identification of potential risks and rewards. Leveraging specific types of privileged information durin…
Reinforcement LearningPathdreamer: A World Model for Indoor Navigation
People navigating in unfamiliar buildings take advantage of myriad visual, spatial and semantic cues to efficiently achieve their navigation goals. Towards equipping computational agents with similar capabilities, we int…
modelSemantic SegmentationVision and Language NavigationDream to Control: Learning Behaviors by Latent Imagination
Learned world models summarize an agent's experience to facilitate learning complex behaviors. While learning world models from high-dimensional sensory inputs is becoming feasible through deep learning, there are many p…
Continuous Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation
This paper investigates training better visual world models for robot manipulation, i.e., models that can predict future visual observations by conditioning on past frames and robot actions. Specifically, we consider wor…
Robot ManipulationSemantic SimilaritySemantic Textual SimilarityVideo PredictionMastering Atari with Discrete World Models
Intelligent agents need to generalize from past experience to achieve goals in complex environments. World models facilitate such generalization and allow learning behaviors from imagined outcomes to increase sample-effi…
Atari GamesGPU