Learning Real-World Robot Policies by Dreaming

2018-05-20 · AJ Piergiovanni, Alan Wu, Michael S. Ryoo

Learning to control robots directly based on images is a primary challenge in robotics. However, many existing reinforcement learning approaches require iteratively obtaining millions of robot samples to learn a policy, which can take significant time. In this paper, we focus on learning a realistic world model capturing the dynamics of scene changes conditioned on robot actions. Our dreaming model can emulate samples equivalent to a sequence of images from the actual environment, technically by learning an action-conditioned future representation/scene regressor. This allows the agent to learn action policies (i.e., visuomotor policies) by interacting with the dreaming model rather than the real-world. We experimentally confirm that our dreaming model enables robot learning of policies that transfer to the real-world.

📄 PDF Abstract BibTeX arXiv:1805.07813

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction

2022-03-01 · Masashi Okada, Tadahiro Taniguchi

The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pix…

Contrastive Learning Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Multi-View Dreaming: Multi-View World Model with Contrastive Learning

2022-03-15 · Akira Kinose, Masashi Okada, Ryo Okumura, Tadahiro Taniguchi

In this paper, we propose Multi-View Dreaming, a novel reinforcement learning agent for integrated recognition and control from multi-view observations by extending Dreaming. Most current reinforcement learning method as…

Contrastive Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization

2024-03-16 · Sai Prasanna, Karim Farid, Raghu Rajan, André Biedenkapp

Zero-shot generalization (ZSG) to unseen dynamics is a major challenge for creating generally capable embodied agents. To address the broader challenge, we start with the simpler setting of contextual reinforcement learn…

Zero-shot Generalization

Learning Versatile Humanoid Manipulation with Touch Dreaming

2026-04-14 · Yaru Niu, Zhenlong Fang, Binghong Chen, Shuai Zhou 외 arxiv

Humanoid robots promise general-purpose assistance, yet real-world humanoid loco-manipulation remains challenging because it requires whole-body stability, end-effector dexterity, and contact-aware interaction under freq…

Nightmare Dreamer: Dreaming About Unsafe States And Planning Ahead

2026-01-08 · Oluwatosin Oseni, Shengjie Wang, Jun Zhu, Micah Corah arxiv

Reinforcement Learning (RL) has shown remarkable success in real-world applications, particularly in robotics control. However, RL adoption remains limited due to insufficient safety guarantees. We introduce Nightmare Dr…

Reinforcement Learning