Intrinsically Motivated Acquisition of Modular Slow Features for Humanoids in Continuous and Non-Stationary Environments

2017-01-17 · Varun Raj Kompella, Laurenz Wiskott

A compact information-rich representation of the environment, also called a feature abstraction, can simplify a robot's task of mapping its raw sensory inputs to useful action sequences. However, in environments that are non-stationary and only partially observable, a single abstraction is probably not sufficient to encode most variations. Therefore, learning multiple sets of spatially or temporally local, modular abstractions of the inputs would be beneficial. How can a robot learn these local abstractions without a teacher? More specifically, how can it decide from where and when to start learning a new abstraction? A recently proposed algorithm called Curious Dr. MISFA addresses this problem. The algorithm is based on two underlying learning principles called artificial curiosity and slowness. The former is used to make the robot self-motivated to explore by rewarding itself whenever it makes progress learning an abstraction; the later is used to update the abstraction by extracting slowly varying components from raw sensory inputs. Curious Dr. MISFA's application is, however, limited to discrete domains constrained by a pre-defined state space and has design limitations that make it unstable in certain situations. This paper presents a significant improvement that is applicable to continuous environments, is computationally less expensive, simpler to use with fewer hyper parameters, and stable in certain non-stationary environments. We demonstrate the efficacy and stability of our method in a vision-based robot simulator.

📄 PDF Abstract BibTeX arXiv:1701.04663

Code (0)

등록된 구현이 없습니다.

Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey

2020-12-17 · Cédric Colas, Tristan Karch, Olivier Sigaud, Pierre-Yves Oudeyer

Building autonomous machines that can explore open-ended environments, discover possible interactions and build repertoires of skills is a general objective of artificial intelligence. Developmental approaches argue that…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

2018-10-15 · Cédric Colas, Pierre Fournier, Olivier Sigaud, Mohamed Chetouani 외

In open-ended environments, autonomous learning agents must set their own goals and build their own curriculum through an intrinsically motivated exploration. They may consider a large diversity of goals, aiming to disco…

Diversity Efficient Exploration Multi-Goal Reinforcement Learning reinforcement-learning+2

Intrinsically Motivated Multimodal Structure Learning

2016-07-15 · Jay Ming Wong, Roderic A. Grupen

We present a long-term intrinsically motivated structure learning method for modeling transition dynamics during controlled interactions between a robot and semi-permanent structures in the world. In particular, we discu…

Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning

2017-08-07 · Sébastien Forestier, Rémy Portelas, Yoan Mollard, Pierre-Yves Oudeyer

Intrinsically motivated spontaneous exploration is a key enabler of autonomous developmental learning in human children. It enables the discovery of skill repertoires through autotelic learning, i.e. the self-generation,…

Developmental Learning Multi-Goal Reinforcement Learning Reinforcement Learning Self-Supervised Learning

RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learning

2024-05-29 · Mingqi Yuan, Roger Creus Castanyer, Bo Li, Xin Jin 외

Extrinsic rewards can effectively guide reinforcement learning (RL) agents in specific tasks. However, extrinsic rewards frequently fall short in complex environments due to the significant human effort needed for their …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)