Reinforcement Learning with Feedback-modulated TD-STDP
Spiking neuron networks have been used successfully to solve simple reinforcement learning tasks with continuous action set applying learning rules based on spike-timing-dependent plasticity (STDP). However, most of these models cannot be applied to reinforcement learning tasks with discrete action set since they assume that the selected action is a deterministic function of firing rate of neurons, which is continuous. In this paper, we propose a new STDP-based learning rule for spiking neuron networks which contains feedback modulation. We show that the STDP-based learning rule can be used to solve reinforcement learning tasks with discrete action set at a speed similar to standard reinforcement learning algorithms when applied to the CartPole and LunarLander tasks. Moreover, we demonstrate that the agent is unable to solve these tasks if feedback modulation is omitted from the learning rule. We conclude that feedback modulation allows better credit assignment when only the units contributing to the executed action and TD error participate in learning.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
First-spike based visual categorization using reward-modulated STDP
Reinforcement learning (RL) has recently regained popularity, with major achievements such as beating the European game of Go champion. Here, for the first time, we show that RL can be used efficiently to train a spiking…
Game of GoObject RecognitionReinforcement LearningReinforcement Learning (RL)Reinforcement Learning in a Neurally Controlled Robot Using Dopamine Modulated STDP
Recent work has shown that dopamine-modulated STDP can solve many of the issues associated with reinforcement learning, such as the distal reward problem. Spiking neural networks provide a useful technique in implementin…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks
The primate visual system has inspired the development of deep artificial neural networks, which have revolutionized the computer vision domain. Yet these networks are much less energy-efficient than their biological cou…
DiagnosticBioLCNet: Reward-modulated Locally Connected Spiking Neural Networks
Brain-inspired computation and information processing alongside compatibility with neuromorphic hardware have made spiking neural networks (SNN) a promising method for solving learning tasks in machine learning (ML). Spi…
Decoderimage-classificationImage ClassificationBiologically Plausible Variational Policy Gradient with Spiking Recurrent Winner-Take-All Networks
One stream of reinforcement learning research is exploring biologically plausible models and algorithms to simulate biological intelligence and fit neuromorphic hardware. Among them, reward-modulated spike-timing-depende…
AllReinforcement Learning (RL)