Noise Facilitation in Associative Memories of Exponential Capacity
Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms have allowed reliable learning and recall of an exponential number of patterns. Although these designs correct external errors in recall, they assume neurons that compute noiselessly, in contrast to the highly variable neurons in brain regions thought to operate associatively such as hippocampus and olfactory cortex. Here we consider associative memories with noisy internal computations and analytically characterize performance. As long as the internal noise level is below a specified threshold, the error probability in the recall phase can be made exceedingly small. More surprisingly, we show that internal noise actually improves the performance of the recall phase while the pattern retrieval capacity remains intact, i.e., the number of stored patterns does not reduce with noise (up to a threshold). Computational experiments lend additional support to our theoretical analysis. This work suggests a functional benefit to noisy neurons in biological neuronal networks.
Code (0)
등록된 구현이 없습니다.
Tasks
HippocampusRetrievalSimilar Papers 제목 키워드 기반
Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels
Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditions of practical inference and biological…
Max-C and Min-D Projection Autoassociative Fuzzy Morphological Memories: Theory and an Application for Face Recognition
Max-C and min-D projection autoassociative fuzzy morphological memories (max-C and min-D PAFMMs) are two layer feedforward fuzzy morphological neural networks able to implement an associative memory designed for the stor…
Face RecognitionRetrievalDense Associative Memory with Epanechnikov Energy
We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum-exponential (LSE) function, LSR is base…
Density EstimationConvolutional Neural Associative Memories: Massive Capacity with Noise Tolerance
The task of a neural associative memory is to retrieve a set of previously memorized patterns from their noisy versions using a network of neurons. An ideal network should have the ability to 1) learn a set of patterns a…
RetrievalA Chaotic Associative Memory
We propose a novel Chaotic Associative Memory model using a network of chaotic Rossler systems and investigate the storage capacity and retrieval capabilities of this model as a function of increasing periodicity and cha…
Retrieval