Unsupervised learning by a nonlinear network with Hebbian excitatory and anti-Hebbian inhibitory neurons
This paper introduces a rate-based nonlinear neural network in which excitatory (E) neurons receive feedforward excitation from sensory (S) neurons, and inhibit each other through disynaptic pathways mediated by inhibitory (I) interneurons. Correlation-based plasticity of disynaptic inhibition serves to incompletely decorrelate E neuron activity, pushing the E neurons to learn distinct sensory features. The plasticity equations additionally contain "extra" terms fostering competition between excitatory synapses converging onto the same postsynaptic neuron and inhibitory synapses diverging from the same presynaptic neuron. The parameters of competition between S$\to$E connections can be adjusted to make learned features look more like "parts" or "wholes." The parameters of competition between I-E connections can be adjusted to set the typical decorrelatedness and sparsity of E neuron activity. Numerical simulations of unsupervised learning show that relatively few I neurons can be sufficient for achieving good decorrelation, and increasing the number of I neurons makes decorrelation more complete. Excitatory and inhibitory inputs to active E neurons are approximately balanced as a result of learning.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Two "correlation games" for a nonlinear network with Hebbian excitatory neurons and anti-Hebbian inhibitory neurons
A companion paper introduces a nonlinear network with Hebbian excitatory (E) neurons that are reciprocally coupled with anti-Hebbian inhibitory (I) neurons and also receive Hebbian feedforward excitation from sensory (S)…
Extended temporal association memory by inhibitory Hebbian learning
Hebbian learning of excitatory synapses plays a central role in storing activity patterns in associative memory models. Furthermore, interstimulus Hebbian learning associates multiple items in the brain by converting tem…
Unsupervised 3D Object Learning through Neuron Activity aware Plasticity
We present an unsupervised deep learning model for 3D object classification. Conventional Hebbian learning, a well-known unsupervised model, suffers from loss of local features leading to reduced performance for tasks wi…
3D Object ClassificationObjectLocal dendritic balance enables learning of efficient representations in networks of spiking neurons
How can neural networks learn to efficiently represent complex and high-dimensional inputs via local plasticity mechanisms? Classical models of representation learning assume that input weights are learned via pairwise H…
Representation LearningDeep Features for CBIR with Scarce Data using Hebbian Learning
Features extracted from Deep Neural Networks (DNNs) have proven to be very effective in the context of Content Based Image Retrieval (CBIR). In recent work, biologically inspired \textit{Hebbian} learning algorithms have…
Content-Based Image RetrievalImage RetrievalRetrievalUnsupervised Pre-training