Supervised and Unsupervised protocols for hetero-associative neural networks
This paper introduces a learning framework for Three-Directional Associative Memory (TAM) models, extending the classical Hebbian paradigm to both supervised and unsupervised protocols within an hetero-associative setting. These neural networks consist of three interconnected layers of binary neurons interacting via generalized Hebbian synaptic couplings that allow learning, storage and retrieval of structured triplets of patterns. By relying upon glassy statistical mechanical techniques (mainly replica theory and Guerra interpolation), we analyze the emergent computational properties of these networks, at work with random (Rademacher) datasets and at the replica-symmetric level of description: we obtain a set of self-consistency equations for the order parameters that quantify the critical dataset sizes (i.e. their thresholds for learning) and describe the retrieval performance of these networks, highlighting the differences between supervised and unsupervised protocols. Numerical simulations validate our theoretical findings and demonstrate the robustness of the captured picture about TAMs also at work with structured datasets. In particular, this study provides insights into the cooperative interplay of layers, beyond that of the neurons within the layers, with potential implications for optimal design of artificial neural network architectures.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Beyond Disorder: Unveiling Cooperativeness in Multidirectional Associative Memories
By leveraging tools from the statistical mechanics of complex systems, in these short notes we extend the architecture of a neural network for hetero-associative memory (called three-directional associative memories, TAM…
RetrievalUnsupervised and Supervised learning by Dense Associative Memory under replica symmetry breaking
Statistical mechanics of spin glasses is one of the main strands toward a comprehension of information processing by neural networks and learning machines. Tackling this approach, at the fairly standard replica symmetric…
Learning Paired-associate Images with An Unsupervised Deep Learning Architecture
This paper presents an unsupervised multi-modal learning system that learns associative representation from two input modalities, or channels, such that input on one channel will correctly generate the associated respons…
Deep LearningStatistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories
We study bi-directional associative neural networks that, exposed to noisy examples of an extensive number of random archetypes, learn the latter (with or without the presence of a teacher) when the supplied information …
Unsupervised Learning using Pretrained CNN and Associative Memory Bank
Deep Convolutional features extracted from a comprehensive labeled dataset, contain substantial representations which could be effectively used in a new domain. Despite the fact that generic features achieved good result…
Few-Shot Image ClassificationFine-Grained Image ClassificationImage ClassificationObject Recognition+1