Multi-scale Transformer Pyramid Networks for Multivariate Time Series Forecasting
Multivariate Time Series (MTS) forecasting involves modeling temporal dependencies within historical records. Transformers have demonstrated remarkable performance in MTS forecasting due to their capability to capture long-term dependencies. However, prior work has been confined to modeling temporal dependencies at either a fixed scale or multiple scales that exponentially increase (most with base 2). This limitation hinders their effectiveness in capturing diverse seasonalities, such as hourly and daily patterns. In this paper, we introduce a dimension invariant embedding technique that captures short-term temporal dependencies and projects MTS data into a higher-dimensional space, while preserving the dimensions of time steps and variables in MTS data. Furthermore, we present a novel Multi-scale Transformer Pyramid Network (MTPNet), specifically designed to effectively capture temporal dependencies at multiple unconstrained scales. The predictions are inferred from multi-scale latent representations obtained from transformers at various scales. Extensive experiments on nine benchmark datasets demonstrate that the proposed MTPNet outperforms recent state-of-the-art methods.
Code (1)
Tasks
Multivariate Time Series ForecastingTime SeriesTime Series ForecastingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PRformer: Pyramidal Recurrent Transformer for Multivariate Time Series Forecasting
The self-attention mechanism in Transformer architecture, invariant to sequence order, necessitates positional embeddings to encode temporal order in time series prediction. We argue that this reliance on positional embe…
Multivariate Time Series ForecastingTemporal SequencesTime SeriesTime Series Forecasting+1DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting
Long-term time series forecasting (LTSF) is hampered by the challenge of modeling complex dependencies that span multiple temporal scales and frequency resolutions. Existing methods, including Transformer and MLP-based m…
Multivariate Time Series ForecastingRepresentation LearningDeep Learning for Multi-Scale Changepoint Detection in Multivariate Time Series
Many real-world time series, such as in health, have changepoints where the system's structure or parameters change. Since changepoints can indicate critical events such as onset of illness, it is highly important to det…
Time SeriesTime Series AnalysisPyramidStyler: Transformer-Based Neural Style Transfer with Pyramidal Positional Encoding and Reinforcement Learning
Neural Style Transfer (NST) has evolved from Gatys et al.'s (2015) CNN-based algorithm, enabling AI-driven artistic image synthesis. However, existing CNN and transformer-based models struggle to scale efficiently to com…
Reinforcement LearningStyle TransferMulti-direction and Multi-scale Pyramid in Transformer for Video-based Pedestrian Retrieval
In video surveillance, pedestrian retrieval (also called person re-identification) is a critical task. This task aims to retrieve the pedestrian of interest from non-overlapping cameras. Recently, transformer-based model…
Person Re-IdentificationRetrieval