Latent World Models For Intrinsically Motivated Exploration

2020-10-05 · NeurIPS 2020 12 · Aleksandr Ermolov, Nicu Sebe

In this work we consider partially observable environments with sparse rewards. We present a self-supervised representation learning method for image-based observations, which arranges embeddings respecting temporal distance of observations. This representation is empirically robust to stochasticity and suitable for novelty detection from the error of a predictive forward model. We consider episodic and life-long uncertainties to guide the exploration. We propose to estimate the missing information about the environment with the world model, which operates in the learned latent space. As a motivation of the method, we analyse the exploration problem in a tabular Partially Observable Labyrinth. We demonstrate the method on image-based hard exploration environments from the Atari benchmark and report significant improvement with respect to prior work. The source code of the method and all the experiments is available at https://github.com/htdt/lwm.

📄 PDF Abstract BibTeX arXiv:2010.02302

Code (1)

htdt/lwm 공식 구현 pytorch

Tasks

Efficient ExplorationNovelty DetectionRepresentation Learning

Unsupervised Learning of Goal Spaces for Intrinsically Motivated Goal Exploration

2018-03-02 · ICLR 2018 1 · Alexandre Péré, Sébastien Forestier, Olivier Sigaud, Pierre-Yves Oudeyer

Intrinsically motivated goal exploration algorithms enable machines to discover repertoires of policies that produce a diversity of effects in complex environments. These exploration algorithms have been shown to allow r…

Diversity Representation Learning

Decoupled Reinforcement Learning to Stabilise Intrinsically-Motivated Exploration

2021-07-19 · ICML Workshop URL 2021 7 · Lukas Schäfer, Filippos Christianos, Josiah P. Hanna, Stefano V. Albrecht

Intrinsic rewards can improve exploration in reinforcement learning, but the exploration process may suffer from instability caused by non-stationary reward shaping and strong dependency on hyperparameters. In this work,…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Intrinsically-Motivated Reinforcement Learning: A Brief Introduction

2022-03-03 · Mingqi Yuan

Reinforcement learning (RL) is one of the three basic paradigms of machine learning. It has demonstrated impressive performance in many complex tasks like Go and StarCraft, which is increasingly involved in smart manufac…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Intrinsically motivated graph exploration using network theories of human curiosity

2023-07-11 · Shubhankar P. Patankar, Mathieu Ouellet, Juan Cervino, Alejandro Ribeiro 외

Intrinsically motivated exploration has proven useful for reinforcement learning, even without additional extrinsic rewards. When the environment is naturally represented as a graph, how to guide exploration best remains…

Graph Neural Network Recommendation Systems reinforcement-learning Reinforcement Learning

Intrinsically Guided Exploration in Meta Reinforcement Learning

2021-01-01 · Jin Zhang, Jianhao Wang, Hao Hu, Tong Chen 외

Deep reinforcement learning algorithms generally require large amounts of data to solve a single task. Meta reinforcement learning (meta-RL) agents learn to adapt to novel unseen tasks with high sample efficiency by extr…

Deep Reinforcement Learning Efficient Exploration Meta Reinforcement Learning MuJoCo+3