Meta-learning within Projective Simulation

2016-02-25 · Adi Makmal, Alexey A. Melnikov, Vedran Dunjko, Hans J. Briegel

Learning models of artificial intelligence can nowadays perform very well on a large variety of tasks. However, in practice different task environments are best handled by different learning models, rather than a single, universal, approach. Most non-trivial models thus require the adjustment of several to many learning parameters, which is often done on a case-by-case basis by an external party. Meta-learning refers to the ability of an agent to autonomously and dynamically adjust its own learning parameters, or meta-parameters. In this work we show how projective simulation, a recently developed model of artificial intelligence, can naturally be extended to account for meta-learning in reinforcement learning settings. The projective simulation approach is based on a random walk process over a network of clips. The suggested meta-learning scheme builds upon the same design and employs clip networks to monitor the agent's performance and to adjust its meta-parameters "on the fly". We distinguish between "reflexive adaptation" and "adaptation through learning", and show the utility of both approaches. In addition, a trade-off between flexibility and learning-time is addressed. The extended model is examined on three different kinds of reinforcement learning tasks, in which the agent has different optimal values of the meta-parameters, and is shown to perform well, reaching near-optimal to optimal success rates in all of them, without ever needing to manually adjust any meta-parameter.

📄 PDF Abstract BibTeX arXiv:1602.08017

Code (0)

등록된 구현이 없습니다.

Tasks

Meta-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Projective simulation with generalization

2015-04-09 · Alexey A. Melnikov, Adi Makmal, Vedran Dunjko, Hans J. Briegel

The ability to generalize is an important feature of any intelligent agent. Not only because it may allow the agent to cope with large amounts of data, but also because in some environments, an agent with no generalizati…

Reinforcement Learning

On the convergence of projective-simulation-based reinforcement learning in Markov decision processes

2019-10-25 · Walter L. Boyajian, Jens Clausen, Lea M. Trenkwalder, Vedran Dunjko 외

In recent years, the interest in leveraging quantum effects for enhancing machine learning tasks has significantly increased. Many algorithms speeding up supervised and unsupervised learning were established. The first f…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Benchmarking projective simulation in navigation problems

2018-04-23 · Alexey A. Melnikov, Adi Makmal, Hans J. Briegel

Projective simulation (PS) is a model for intelligent agents with a deliberation capacity that is based on episodic memory. The model has been shown to provide a flexible framework for constructing reinforcement-learning…

Benchmarking Q-Learning reinforcement-learning Reinforcement Learning+1

A Projective Simulation Scheme for Partially-Observable Multi-Agent Systems

2016-10-29 · Rasoul Kheiri

We introduce a kind of partial observability to the projective simulation (PS) learning method. It is done by adding a belief projection operator and an observability parameter to the original framework of the efficiency…

Projective Subspace Networks For Few-Shot Learning

2019-05-01 · ICLR 2019 5 · Christian Simon, Piotr Koniusz, Mehrtash Harandi

Generalization from limited examples, usually studied under the umbrella of meta-learning, equips learning techniques with the ability to adapt quickly in dynamical environments and proves to be an essential aspect of li…

Few-Shot Learning General Classification Lifelong learning Meta-Learning