The Predictive Forward-Forward Algorithm
We propose the predictive forward-forward (PFF) algorithm for conducting credit assignment in neural systems. Specifically, we design a novel, dynamic recurrent neural system that learns a directed generative circuit jointly and simultaneously with a representation circuit. Notably, the system integrates learnable lateral competition, noise injection, and elements of predictive coding, an emerging and viable neurobiological process theory of cortical function, with the forward-forward (FF) adaptation scheme. Furthermore, PFF efficiently learns to propagate learning signals and updates synapses with forward passes only, eliminating key structural and computational constraints imposed by backpropagation-based schemes. Besides computational advantages, the PFF process could prove useful for understanding the learning mechanisms behind biological neurons that use local signals despite missing feedback connections. We run experiments on image data and demonstrate that the PFF procedure works as well as backpropagation, offering a promising brain-inspired algorithm for classifying, reconstructing, and synthesizing data patterns.
Code (1)
Similar Papers 제목 키워드 기반
Unifying back-propagation and forward-forward algorithms through model predictive control
We introduce a Model Predictive Control (MPC) framework for training deep neural networks, systematically unifying the Back-Propagation (BP) and Forward-Forward (FF) algorithms. At the same time, it gives rise to a range…
Model Predictive ControlUnsupervised inference for optimizing deep feedforward neural network architecture
We propose an unsupervised inference algorithm to find optimal deep feedforward neural network architecture by modeling hierarchical representations of given data set. Our algorithm learns the optimal neural network arch…
Concentration and Calibration in Predictive Bayesian Inference
Predictive Bayesian inference (PBI) represents a model-and prior-agnostic approach to standard Bayesian inference which allows users to quantify uncertainty for a functional of interest only by specifying a forward predi…
Bayesian InferenceMetaheuristics optimized feedforward neural networks for efficient stock price prediction
The prediction of stock prices is an important task in economics, investment and making financial decisions. This has, for decades, spurred the interest of many researchers to make focused contributions to the design of …
Decision MakingStock Price PredictionTime SeriesTime Series AnalysisBenefits of Feedforward for Model Predictive Airpath Control of Diesel Engines
This paper investigates options to complement a diesel engine airpath feedback controller with a feedforward. The control objective is to track the intake manifold pressure and exhaust gas recirculation (EGR) rate target…
Model Predictive Control