An Attention Free Long Short-Term Memory for Time Series Forecasting
Deep learning is playing an increasingly important role in time series analysis. We focused on time series forecasting using attention free mechanism, a more efficient framework, and proposed a new architecture for time series prediction for which linear models seem to be unable to capture the time dependence. We proposed an architecture built using attention free LSTM layers that overcome linear models for conditional variance prediction. Our findings confirm the validity of our model, which also allowed to improve the prediction capacity of a LSTM, while improving the efficiency of the learning task.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionTime SeriesTime Series AnalysisTime Series ForecastingTime Series PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Short window attention enables long-term memorization
Recent works show that hybrid architectures combining local sliding window attention layers and global attention layers outperform either of these architectures taken separately. However, the impact of the window length …
Robust offset-free constrained Model Predictive Control with Long Short-Term Memory Networks -- Extended version
This paper develops a control scheme, based on the use of Long Short-Term Memory neural network models and Nonlinear Model Predictive Control, which guarantees recursive feasibility with slow time variant set-points and …
Model Predictive ControlMELODI: Exploring Memory Compression for Long Contexts
We present MELODI, a novel memory architecture designed to efficiently process long documents using short context windows. The key principle behind MELODI is to represent short-term and long-term memory as a hierarchical…
Japanese Sentiment Classification using a Tree-Structured Long Short-Term Memory with Attention
Previous approaches to training syntax-based sentiment classification models required phrase-level annotated corpora, which are not readily available in many languages other than English. Thus, we propose the use of tree…
ClassificationGeneral ClassificationSentiment AnalysisSentiment ClassificationLong Short-Term Attention
Attention is an important cognition process of humans, which helps humans concentrate on critical information during their perception and learning. However, although many machine learning models can remember information …