Learning and processing the ordinal information of temporal sequences in recurrent neural circuits
Temporal sequence processing is fundamental in brain cognitive functions. Experimental data has indicated that the representations of ordinal information and contents of temporal sequences are disentangled in the brain, but the neural mechanism underlying this disentanglement remains largely unclear. Here, we investigate how recurrent neural circuits learn to represent the abstract order structure of temporal sequences, and how this disentangled representation of order structure from that of contents facilitates the processing of temporal sequences. We show that with an appropriate learn protocol, a recurrent neural circuit can learn a set of tree-structured attractor states to encode the corresponding tree-structured orders of given temporal sequences. This abstract temporal order template can then be bound with different contents, allowing for flexible and robust temporal sequence processing. Using a transfer learning task, we demonstrate that the reuse of a temporal order template facilitates the acquisition of new temporal sequences of the same or similar ordinal structure. Using a key-word spotting task, we demonstrate that the attractor representation of order structure improves the robustness of temporal sequence discrimination, if the ordinal information is the key to differentiate different sequences. We hope this study gives us insights into the neural mechanism of representing the ordinal information of temporal sequences in the brain, and helps us to develop brain-inspired temporal sequence processing algorithms.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Learning Sequence Attractors in Recurrent Networks with Hidden Neurons
The brain is targeted for processing temporal sequence information. It remains largely unclear how the brain learns to store and retrieve sequence memories. Here, we study how recurrent networks of binary neurons learn s…
Multi-instance Dynamic Ordinal Random Fields for Weakly-Supervised Pain Intensity Estimation
In this paper, we address the Multi-Instance-Learning (MIL) problem when bag labels are naturally represented as ordinal variables (Multi--Instance--Ordinal Regression). Moreover, we consider the case where bags are temp…
Temporal SequencesSpaces and sequences in the hippocampus: a homological perspective
Topological techniques have become a popular tool for studying information flows in neural networks. In particular, simplicial homology theory is used to analyze how cognitive representations of space emerge from large c…
HippocampusLearning The Sequential Temporal Information with Recurrent Neural Networks
Recurrent Networks are one of the most powerful and promising artificial neural network algorithms to processing the sequential data such as natural languages, sound, time series data. Unlike traditional feed-forward net…
Image CaptioningLanguage ModelingLanguage ModellingMachine Translation+8Time-Warping Invariant Quantum Recurrent Neural Networks via Quantum-Classical Adaptive Gating
Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNN), as it facilitates retention of past information necessary to predict the future, providing a mechanism that pres…