paper-with-me

Papers

FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting

2022-01-30 · Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, Rong Jin

Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e.g. overall trend). To address these problems, we propose to combine Transformer with the seasonal-trend decomposition method, in which the decomposition method captures the global profile of time series while Transformers capture more detailed structures. To further enhance the performance of Transformer for long-term prediction, we exploit the fact that most time series tend to have a sparse representation in well-known basis such as Fourier transform, and develop a frequency enhanced Transformer. Besides being more effective, the proposed method, termed as Frequency Enhanced Decomposed Transformer ({\bf FEDformer}), is more efficient than standard Transformer with a linear complexity to the sequence length. Our empirical studies with six benchmark datasets show that compared with state-of-the-art methods, FEDformer can reduce prediction error by $14.8\%$ and $22.6\%$ for multivariate and univariate time series, respectively. Code is publicly available at https://github.com/MAZiqing/FEDformer.

📄 PDF Abstract BibTeX arXiv:2201.12740

Code (3)

WenjieDu/PyPOTS 공식 구현 pytorch
MAZiqing/FEDformer pytorch
damo-di-ml/icml2022-fedformer pytorch

Tasks

Time SeriesTime Series Analysis

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Residual Connection 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Similar Papers 제목 키워드 기반

A FEDformer-Based Hybrid Framework for Anomaly Detection and Risk Forecasting in Financial Time Series

2025-11-17 · Ziling Fan, Ruijia Liang, Yiwen Hu arxiv

Financial markets are inherently volatile and prone to sudden disruptions such as market crashes, flash collapses, and liquidity crises. Accurate anomaly detection and early risk forecasting in financial time series are …

Anomaly Detection

TFDNet: Time-Frequency Enhanced Decomposed Network for Long-term Time Series Forecasting

2023-08-25 · Yuxiao Luo, Ziyu Lyu, Xingyu Huang

Long-term time series forecasting is a vital task and has a wide range of real applications. Recent methods focus on capturing the underlying patterns from one single domain (e.g. the time domain or the frequency domain)…

Time SeriesTime Series Forecasting

FedFormer: Contextual Federation with Attention in Reinforcement Learning

2022-05-27 · Liam Hebert, Lukasz Golab, Pascal Poupart, Robin Cohen

A core issue in multi-agent federated reinforcement learning is defining how to aggregate insights from multiple agents. This is commonly done by taking the average of each participating agent's model weights into one co…

Federated Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)

TwinFormer: A Dual-Level Transformer for Long-Sequence Time-Series Forecasting

2025-12-13 · Mahima Kumavat, Aditya Maheshwari arxiv

TwinFormer is a hierarchical Transformer for long-sequence time-series forecasting. It divides the input into non-overlapping temporal patches and processes them in two stages: (1) a Local Informer with top-$k$ Sparse At…

DETNO: A Diffusion-Enhanced Transformer Neural Operator for Long-Term Traffic Forecasting

2025-08-26 · Owais Ahmad, Milad Ramezankhani, Anirudh Deodhar arxiv

Accurate long-term traffic forecasting remains a critical challenge in intelligent transportation systems, particularly when predicting high-frequency traffic phenomena such as shock waves and congestion boundaries over …