Biologically Motivated Algorithms for Propagating Local Target Representations
Finding biologically plausible alternatives to back-propagation of errors is a fundamentally important challenge in artificial neural network research. In this paper, we propose a learning algorithm called error-driven Local Representation Alignment (LRA-E), which has strong connections to predictive coding, a theory that offers a mechanistic way of describing neurocomputational machinery. In addition, we propose an improved variant of Difference Target Propagation, another procedure that comes from the same family of algorithms as LRA-E. We compare our procedures to several other biologically-motivated algorithms, including two feedback alignment algorithms and Equilibrium Propagation. In two benchmarks, we find that both of our proposed algorithms yield stable performance and strong generalization compared to other competing back-propagation alternatives when training deeper, highly nonlinear networks, with LRA-E performing the best overall.
Code (1)
Similar Papers 제목 키워드 기반
Biologically-Motivated Deep Learning Method using Hierarchical Competitive Learning
This study proposes a novel biologically-motivated learning method for deep convolutional neural networks (CNNs). The combination of CNNs and back propagation (BP) learning is the most powerful method in recent machine l…
Deep LearningTime SeriesTime Series AnalysisAssessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
The backpropagation of error algorithm (BP) is impossible to implement in a real brain. The recent success of deep networks in machine learning and AI, however, has inspired proposals for understanding how the brain migh…
Deep LearningAdaptive Bidirectional Backpropagation: Towards Biologically Plausible Error Signal Transmission in Neural Networks
The back-propagation (BP) algorithm has been considered the de-facto method for training deep neural networks. It back-propagates errors from the output layer to the hidden layers in an exact manner using the transpose o…
Are skip connections necessary for biologically plausible learning rules?
Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these met…
Gradient-adjusted Incremental Target Propagation Provides Effective Credit Assignment in Deep Neural Networks
Many of the recent advances in the field of artificial intelligence have been fueled by the highly successful backpropagation of error (BP) algorithm, which efficiently solves the credit assignment problem in artificial …
Handwritten Digit Recognition