CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

2018-10-15 · Cédric Colas, Pierre Fournier, Olivier Sigaud, Mohamed Chetouani, Pierre-Yves Oudeyer

In open-ended environments, autonomous learning agents must set their own goals and build their own curriculum through an intrinsically motivated exploration. They may consider a large diversity of goals, aiming to discover what is controllable in their environments, and what is not. Because some goals might prove easy and some impossible, agents must actively select which goal to practice at any moment, to maximize their overall mastery on the set of learnable goals. This paper proposes CURIOUS, an algorithm that leverages 1) a modular Universal Value Function Approximator with hindsight learning to achieve a diversity of goals of different kinds within a unique policy and 2) an automated curriculum learning mechanism that biases the attention of the agent towards goals maximizing the absolute learning progress. Agents focus sequentially on goals of increasing complexity, and focus back on goals that are being forgotten. Experiments conducted in a new modular-goal robotic environment show the resulting developmental self-organization of a learning curriculum, and demonstrate properties of robustness to distracting goals, forgetting and changes in body properties.

📄 PDF Abstract BibTeX arXiv:1810.06284

Code (1)

flowersteam/curious 공식 구현

Tasks

DiversityEfficient ExplorationMulti-Goal Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

DDPG DDPG, or Deep Deterministic Policy Gradient, is an actor-critic, model-free algorithm based on the deterministic policy gradient that can operate over continuous action…

COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Exploration

2019-05-22 · Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher P. Burgess 외

Data efficiency and robustness to task-irrelevant perturbations are long-standing challenges for deep reinforcement learning algorithms. Here we introduce a modular approach to addressing these challenges in a continuous…

continuous-control Continuous Control Deep Reinforcement Learning Object+4

Intrinsically Motivated Acquisition of Modular Slow Features for Humanoids in Continuous and Non-Stationary Environments

2017-01-17 · Varun Raj Kompella, Laurenz Wiskott

A compact information-rich representation of the environment, also called a feature abstraction, can simplify a robot's task of mapping its raw sensory inputs to useful action sequences. However, in environments that are…

Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

2017-08-07 · Sébastien Forestier, Rémy Portelas, Yoan Mollard, Pierre-Yves Oudeyer

Intrinsically motivated spontaneous exploration is a key enabler of autonomous developmental learning in human children. It enables the discovery of skill repertoires through autotelic learning, i.e. the self-generation,…

Developmental Learning Multi-Goal Reinforcement Learning Reinforcement Learning Self-Supervised Learning

Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey

2020-12-17 · Cédric Colas, Tristan Karch, Olivier Sigaud, Pierre-Yves Oudeyer

Building autonomous machines that can explore open-ended environments, discover possible interactions and build repertoires of skills is a general objective of artificial intelligence. Developmental approaches argue that…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Boredom-driven curious learning by Homeo-Heterostatic Value Gradients

2018-06-05 · Yen Yu, Acer Y. C. Chang, Ryota Kanai

This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward m…