Lifelong Reinforcement Learning via Neuromodulation
Navigating multiple tasks$\unicode{x2014}$for instance in succession as in continual or lifelong learning, or in distributions as in meta or multi-task learning$\unicode{x2014}$requires some notion of adaptation. Evolution over timescales of millennia has imbued humans and other animals with highly effective adaptive learning and decision-making strategies. Central to these functions are so-called neuromodulatory systems. In this work we introduce an abstract framework for integrating theories and evidence from neuroscience and the cognitive sciences into the design of adaptive artificial reinforcement learning algorithms. We give a concrete instance of this framework built on literature surrounding the neuromodulators Acetylcholine (ACh) and Noradrenaline (NA), and empirically validate the effectiveness of the resulting adaptive algorithm in a non-stationary multi-armed bandit problem. We conclude with a theory-based experiment proposal providing an avenue to link our framework back to efforts in experimental neuroscience.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingLifelong learningMulti-Task Learningreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
The impressive lifelong learning in animal brains is primarily enabled by plastic changes in synaptic connectivity. Importantly, these changes are not passive, but are actively controlled by neuromodulation, which is its…
Language ModelingLanguage ModellingLifelong learningreinforcement-learning+2Introducing Neuromodulation in Deep Neural Networks to Learn Adaptive Behaviours
Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intellig…
Meta Reinforcement LearningReinforcement LearningSome Insights into Lifelong Reinforcement Learning Systems
A lifelong reinforcement learning system is a learning system that has the ability to learn through trail-and-error interaction with the environment over its lifetime. In this paper, I give some arguments to show that th…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Context Meta-Reinforcement Learning via Neuromodulation
Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt quickly to tasks from few samples in dynamic environments. Such a feat is achieved through dynamic representations in an agent's policy network (obt…
continuous-controlContinuous ControlMeta Reinforcement Learningreinforcement-learning+2Rapid online learning and robust recall in a neuromorphic olfactory circuit
We present a neural algorithm for the rapid online learning and identification of odorant samples under noise, based on the architecture of the mammalian olfactory bulb and implemented on the Intel Loihi neuromorphic sys…
Lifelong learning