A memory enhanced LSTM for modeling complex temporal dependencies
In this paper, we present Gamma-LSTM, an enhanced long short term memory (LSTM) unit, to enable learning of hierarchical representations through multiple stages of temporal abstractions. Gamma memory, a hierarchical memory unit, forms the central memory of Gamma-LSTM with gates to regulate the information flow into various levels of hierarchy, thus providing the unit with a control to pick the appropriate level of hierarchy to process the input at a given instant of time. We demonstrate better performance of Gamma-LSTM model regular and stacked LSTMs in two settings (pixel-by-pixel MNIST digit classification and natural language inference) placing emphasis on the ability to generalize over long sequences.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural Language InferenceSimilar Papers 제목 키워드 기반
Quantum Kernel-Based Long Short-term Memory
The integration of quantum computing into classical machine learning architectures has emerged as a promising approach to enhance model efficiency and computational capacity. In this work, we introduce the Quantum Kernel…
Edge-computingQuantum Machine LearningX-TRACK: Physics-Aware xLSTM for Realistic Vehicle Trajectory Prediction
Accurate trajectory prediction is crucial for safe and reliable autonomous driving systems, requiring models that capture long-term temporal dependencies while accounting for social interactions among neighboring vehicle…
Trajectory PredictionAutonomous DrivingStoxLSTM: A Stochastic Extended Long Short-Term Memory Network for Time Series Forecasting
The Extended Long Short-Term Memory (xLSTM) network has demonstrated strong capability in modeling complex long-term dependencies in time series data. Despite its success, the deterministic architecture of xLSTM limits i…
Time Series ForecastingMemory-DD: A Low-Complexity Dendrite-Inspired Neuron for Temporal Prediction Tasks
Dendrite-inspired neurons have been widely used in tasks such as image classification due to low computational complexity and fast inference speed. Temporal data prediction, as a key machine learning task, plays a key ro…
Image ClassificationTemporal SequencesData-Driven Predictive Modeling of Neuronal Dynamics using Long Short-Term Memory
Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a …