Eligibility traces provide a data-inspired alternative to backpropagation through time
Learning in recurrent neural networks (RNNs) is most often implemented by gradient descent using backpropagation through time (BPTT), but BPTT does not model accurately how the brain learns. Instead, many experimental results on synaptic plasticity can be summarized as three-factor learning rules involving eligibility traces of the local neural activity and a third factor. We present here eligibility propagation (e-prop), a new factorization of the loss gradients in RNNs that fits the framework of three factor learning rules when derived for biophysical spiking neuron models. When tested on the TIMIT speech recognition benchmark, it is competitive with BPTT both for training artificial LSTM networks and spiking RNNs. Further analysis suggests that the diversity of learning signals and the consideration of slow internal neural dynamics are decisive to the learning efficiency of e-prop.
Code (0)
등록된 구현이 없습니다.
Tasks
Diversityspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Learning From the Past with Cascading Eligibility Traces
Animals often receive information about errors and rewards after a significant delay. For example, there is typically a delay of tens to hundreds of milliseconds between motor actions and visual feedback. The standard ap…
Investigating Recurrence and Eligibility Traces in Deep Q-Networks
Eligibility traces in reinforcement learning are used as a bias-variance trade-off and can often speed up training time by propagating knowledge back over time-steps in a single update. We investigate the use of eligibil…
Atari Gamesreinforcement-learningReinforcement LearningReinforcement Learning (RL)Expected Eligibility Traces
The question of how to determine which states and actions are responsible for a certain outcome is known as the credit assignment problem and remains a central research question in reinforcement learning and artificial i…
counterfactualEnhanced-FQL($λ$), an Efficient and Interpretable RL with novel Fuzzy Eligibility Traces and Segmented Experience Replay
This paper introduces a fuzzy reinforcement learning framework, Enhanced-FQL($λ$), that integrates novel Fuzzified Eligibility Traces (FET) and Segmented Experience Replay (SER) into fuzzy Q-learning with the Fuzzified B…
Reinforcement LearningContinuous ControlAdaptive and Multiple Time-scale Eligibility Traces for Online Deep Reinforcement Learning
Deep reinforcement learning (DRL) is one promising approach to teaching robots to perform complex tasks. Because methods that directly reuse the stored experience data cannot follow the change of the environment in robot…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)