paper-with-me

Papers

Inter-Series Transformer: Attending to Products in Time Series Forecasting

2024-08-07 · Rares Cristian, Pavithra Harsha, Clemente Ocejo, Georgia Perakis, Brian Quanz, Ioannis Spantidakis, Hamza Zerhouni

Time series forecasting is an important task in many fields ranging from supply chain management to weather forecasting. Recently, Transformer neural network architectures have shown promising results in forecasting on common time series benchmark datasets. However, application to supply chain demand forecasting, which can have challenging characteristics such as sparsity and cross-series effects, has been limited. In this work, we explore the application of Transformer-based models to supply chain demand forecasting. In particular, we develop a new Transformer-based forecasting approach using a shared, multi-task per-time series network with an initial component applying attention across time series, to capture interactions and help address sparsity. We provide a case study applying our approach to successfully improve demand prediction for a medical device manufacturing company. To further validate our approach, we also apply it to public demand forecasting datasets as well and demonstrate competitive to superior performance compared to a variety of baseline and state-of-the-art forecast methods across the private and public datasets.

📄 PDF Abstract BibTeX arXiv:2408.03872

Code (0)

등록된 구현이 없습니다.

Tasks

Demand ForecastingTime SeriesTime Series ForecastingWeather Forecasting

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Residual Connection 설명 없음
Multi-Head Attention 설명 없음
Attention 설명 없음
Position-Wise Feed-Forward Layer 설명 없음
Adam 설명 없음
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Similar Papers 제목 키워드 기반

Using Pre-trained LLMs for Multivariate Time Series Forecasting

2025-01-10 · Malcolm L. Wolff, Shenghao Yang, Kari Torkkola, Michael W. Mahoney

Pre-trained Large Language Models (LLMs) encapsulate large amounts of knowledge and take enormous amounts of compute to train. We make use of this resource, together with the observation that LLMs are able to transfer kn…

DecoderMultivariate Time Series ForecastingTime SeriesTime Series Forecasting

S2TX: Cross-Attention Multi-Scale State-Space Transformer for Time Series Forecasting

2025-02-17 · Zihao Wu, Juncheng Dong, Haoming Yang, Vahid Tarokh

Time series forecasting has recently achieved significant progress with multi-scale models to address the heterogeneity between long and short range patterns. Despite their state-of-the-art performance, we identify two p…

MambaTime SeriesTime Series Forecasting

Transformers for scientific data: a pedagogical review for astronomers

2023-10-18 · Dimitrios Tanoglidis, Bhuvnesh Jain, Helen Qu

The deep learning architecture associated with ChatGPT and related generative AI products is known as transformers. Initially applied to Natural Language Processing, transformers and the self-attention mechanism they exp…

AstronomyTime Series

Attending to Emotional Narratives

2019-07-08 · Zhengxuan Wu, Xiyu Zhang, Tan Zhi-Xuan, Jamil Zaki 외

Attention mechanisms in deep neural networks have achieved excellent performance on sequence-prediction tasks. Here, we show that these recently-proposed attention-based mechanisms---in particular, the Transformer with i…

Emotion RecognitionTime SeriesTime Series Analysis

SepTr: Separable Transformer for Audio Spectrogram Processing

2022-03-17 · Nicolae-Catalin Ristea, Radu Tudor Ionescu, Fahad Shahbaz Khan

Following the successful application of vision transformers in multiple computer vision tasks, these models have drawn the attention of the signal processing community. This is because signals are often represented as sp…

Audio ClassificationSpeech Emotion RecognitionTime Series Analysis