paper-with-me

Papers

DSSRNN: Decomposition-Enhanced State-Space Recurrent Neural Network for Time-Series Analysis

2024-12-01 · Ahmad Mohammadshirazi, Ali Nosratifiroozsalari, Rajiv Ramnath

Time series forecasting is a crucial yet challenging task in machine learning, requiring domain-specific knowledge due to its wide-ranging applications. While recent Transformer models have improved forecasting capabilities, they come with high computational costs. Linear-based models have shown better accuracy than Transformers but still fall short of ideal performance. To address these challenges, we introduce the Decomposition State-Space Recurrent Neural Network (DSSRNN), a novel framework designed for both long-term and short-term time series forecasting. DSSRNN uniquely combines decomposition analysis to capture seasonal and trend components with state-space models and physics-based equations. We evaluate DSSRNN's performance on indoor air quality datasets, focusing on CO2 concentration prediction across various forecasting horizons. Results demonstrate that DSSRNN consistently outperforms state-of-the-art models, including transformer-based architectures, in terms of both Mean Squared Error (MSE) and Mean Absolute Error (MAE). For example, at the shortest horizon (T=96) in Office 1, DSSRNN achieved an MSE of 0.378 and an MAE of 0.401, significantly lower than competing models. Additionally, DSSRNN exhibits superior computational efficiency compared to more complex models. While not as lightweight as the DLinear model, DSSRNN achieves a balance between performance and efficiency, with only 0.11G MACs and 437MiB memory usage, and an inference time of 0.58ms for long-term forecasting. This work not only showcases DSSRNN's success but also establishes a new benchmark for physics-informed machine learning in environmental forecasting and potentially other domains.

📄 PDF Abstract BibTeX arXiv:2412.00994

Code (1)

ahmad-shirazi/DSSRNN 공식 구현 pytorch

Tasks

Computational EfficiencyPhysics-informed machine learningState Space ModelsTime SeriesTime Series AnalysisTime Series Forecasting

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Residual Connection 설명 없음
Adam 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
MAE 설명 없음

Similar Papers 제목 키워드 기반

State-space Decomposition Model for Video Prediction Considering Long-term Motion Trend

2024-04-17 · Fei Cui, Jiaojiao Fang, Xiaojiang Wu, Zelong Lai 외

Stochastic video prediction enables the consideration of uncertainty in future motion, thereby providing a better reflection of the dynamic nature of the environment. Stochastic video prediction methods based on image au…

motion predictionPredictionState Space ModelsVideo Prediction

Hybrid Recurrent Models Support Emergent Descriptions for Hierarchical Planning and Control

2024-08-20 · Poppy Collis, Ryan Singh, Paul F Kinghorn, Christopher L Buckley

An open problem in artificial intelligence is how systems can flexibly learn discrete abstractions that are useful for solving inherently continuous problems. Previous work has demonstrated that a class of hybrid state-s…

Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels

2021-11-07 · Zexi Hu, Xiaoming Chen, Henry Wing Fung Yeung, Yuk Ying Chung 외

Despite the recent progress in light field super-resolution (LFSR) achieved by convolutional neural networks, the correlation information of light field (LF) images has not been sufficiently studied and exploited due to …

Material RecognitionSuper-Resolution

Fourier-Enhanced Recurrent Neural Networks for Electrical Load Time Series Downscaling

2025-11-27 · Qi Chen, Mihai Anitescu arxiv

We present a Fourier-enhanced recurrent neural network (RNN) for downscaling electrical loads. The model combines (i) a recurrent backbone driven by low-resolution inputs, (ii) explicit Fourier seasonal embeddings fused …

Dynamic Analysis and an Eigen Initializer for Recurrent Neural Networks

2023-07-28 · Ran Dou, Jose Principe

In recurrent neural networks, learning long-term dependency is the main difficulty due to the vanishing and exploding gradient problem. Many researchers are dedicated to solving this issue and they proposed many algorith…

Machine Translation