Transfer Learning with Sparse Associative Memories
In this paper, we introduce a novel layer designed to be used as the output of pre-trained neural networks in the context of classification. Based on Associative Memories, this layer can help design Deep Neural Networks which support incremental learning and that can be (partially) trained in real time on embedded devices. Experiments on the ImageNet dataset and other different domain specific datasets show that it is possible to design more flexible and faster-to-train Neural Networks at the cost of a slight decrease in accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationIncremental LearningTransfer LearningSimilar Papers 제목 키워드 기반
Storing non-uniformly distributed messages in networks of neural cliques
Associative memories are data structures that allow retrieval of stored messages from part of their content. They thus behave similarly to human brain that is capable for instance of retrieving the end of a song given it…
RetrievalAssociative Memories Based on Multiple-Valued Sparse Clustered Networks
Associative memories are structures that store data patterns and retrieve them given partial inputs. Sparse Clustered Networks (SCNs) are recently-introduced binary-weighted associative memories that significantly improv…
RetrievalA study of retrieval algorithms of sparse messages in networks of neural cliques
Associative memories are data structures addressed using part of the content rather than an index. They offer good fault reliability and biological plausibility. Among different families of associative memories, sparse o…
RetrievalRobust Associative Memories Naturally Occuring From Recurrent Hebbian Networks Under Noise
The brain is a noisy system subject to energy constraints. These facts are rarely taken into account when modelling artificial neural networks. In this paper, we are interested in demonstrating that those factors can act…
Quantization robustness from dense representations of sparse functions in high-capacity kernel associative memory
High-capacity associative memories based on Kernel Logistic Regression (KLR) achieve strong retrieval performance but typically require substantial computational resources. This paper investigates the compressibility of …