Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction

2020-07-29 · Masashi Okada, Tadahiro Taniguchi

In the present paper, we propose a decoder-free extension of Dreamer, a leading model-based reinforcement learning (MBRL) method from pixels. Dreamer is a sample- and cost-efficient solution to robot learning, as it is used to train latent state-space models based on a variational autoencoder and to conduct policy optimization by latent trajectory imagination. However, this autoencoding based approach often causes object vanishing, in which the autoencoder fails to perceives key objects for solving control tasks, and thus significantly limiting Dreamer's potential. This work aims to relieve this Dreamer's bottleneck and enhance its performance by means of removing the decoder. For this purpose, we firstly derive a likelihood-free and InfoMax objective of contrastive learning from the evidence lower bound of Dreamer. Secondly, we incorporate two components, (i) independent linear dynamics and (ii) the random crop data augmentation, to the learning scheme so as to improve the training performance. In comparison to Dreamer and other recent model-free reinforcement learning methods, our newly devised Dreamer with InfoMax and without generative decoder (Dreaming) achieves the best scores on 5 difficult simulated robotics tasks, in which Dreamer suffers from object vanishing.

📄 PDF Abstract BibTeX arXiv:2007.14535

Code (0)

등록된 구현이 없습니다.

Tasks

Contrastive Learning Data Augmentation DecoderModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)State Space Models

Methods 이 논문이 사용한 방법론

Solana Customer Service Number +1-833-534-1729 설명 없음

DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction

2022-03-01 · Masashi Okada, Tadahiro Taniguchi

The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pix…

Contrastive Learning Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness

2026-03-22 · Ziou Hu, Xiangtong Yao, Yuan Meng, Zhenshan Bing 외 arxiv

Diffusion policies excel at visuomotor control but often fail catastrophically under severe out-of-distribution (OOD) disturbances, such as unexpected object displacements or visual corruptions. To address this vulnerabi…

Multi-View Dreaming: Multi-View World Model with Contrastive Learning

2022-03-15 · Akira Kinose, Masashi Okada, Ryo Okumura, Tadahiro Taniguchi

In this paper, we propose Multi-View Dreaming, a novel reinforcement learning agent for integrated recognition and control from multi-view observations by extending Dreaming. Most current reinforcement learning method as…

Contrastive Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination

2023-11-11 · Lu Wen, Songan Zhang, H. Eric Tseng, Huei Peng

Meta reinforcement learning (Meta RL) has been amply explored to quickly learn an unseen task by transferring previously learned knowledge from similar tasks. However, most state-of-the-art algorithms require the meta-tr…

Meta Reinforcement Learning

Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming

2024-02-29 · Hany Hamed, Subin Kim, Dongyeong Kim, Jaesik Yoon 외

Model-based reinforcement learning (MBRL) has been a primary approach to ameliorating the sample efficiency issue as well as to make a generalist agent. However, there has not been much effort toward enhancing the strate…

Model-based Reinforcement Learning