paper-with-me

홈 › Papers

From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting

2024-03-17 · Zhen Zeng, Rachneet Kaur, Suchetha Siddagangappa, Tucker Balch, Manuela Veloso

Time series forecasting plays a crucial role in decision-making across various domains, but it presents significant challenges. Recent studies have explored image-driven approaches using computer vision models to address these challenges, often employing lineplots as the visual representation of time series data. In this paper, we propose a novel approach that uses time-frequency spectrograms as the visual representation of time series data. We introduce the use of a vision transformer for multimodal learning, showcasing the advantages of our approach across diverse datasets from different domains. To evaluate its effectiveness, we compare our method against statistical baselines (EMA and ARIMA), a state-of-the-art deep learning-based approach (DeepAR), other visual representations of time series data (lineplot images), and an ablation study on using only the time series as input. Our experiments demonstrate the benefits of utilizing spectrograms as a visual representation for time series data, along with the advantages of employing a vision transformer for simultaneous learning in both the time and frequency domains.

📄 PDF Abstract BibTeX arXiv:2403.11047

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingTime SeriesTime Series Forecasting

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Multi-Head Attention 설명 없음
Residual Connection 설명 없음
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…
Vision Transformer The Vision Transformer, or ViT, is a model for image classification that employs a Transformer-like architecture over…

Similar Papers 제목 키워드 기반

SepTr: Separable Transformer for Audio Spectrogram Processing

2022-03-17 · Nicolae-Catalin Ristea, Radu Tudor Ionescu, Fahad Shahbaz Khan

Following the successful application of vision transformers in multiple computer vision tasks, these models have drawn the attention of the signal processing community. This is because signals are often represented as sp…

Audio ClassificationSpeech Emotion RecognitionTime Series Analysis

Stress Classification from ECG Signals Using Vision Transformer

2026-03-19 · Zeeshan Ahmad, Naimul Khan arxiv

Vision Transformers have shown tremendous success in numerous computer vision applications; however, they have not been exploited for stress assessment using physiological signals such as Electrocardiogram (ECG). In orde…

Binary Classification

Transformer Based Machine Fault Detection From Audio Input

2026-04-14 · Kiran Voderhobli Holla arxiv

In recent years, Sound AI is being increasingly used to predict machine failures. By attaching a microphone to the machine of interest, one can get real time data on machine behavior from the field. Traditionally, Convol…

Full-Frequency Temporal Patching and Structured Masking for Enhanced Audio Classification

2025-08-28 · Aditya Makineni, Baocheng Geng, Qing Tian arxiv

Transformers and State-Space Models (SSMs) have advanced audio classification by modeling spectrograms as sequences of patches. However, existing models such as the Audio Spectrogram Transformer (AST) and Audio Mamba (Au…

Audio Classification

SSAST: Self-Supervised Audio Spectrogram Transformer

2021-10-19 · Yuan Gong, Cheng-I Jeff Lai, Yu-An Chung, James Glass

Recently, neural networks based purely on self-attention, such as the Vision Transformer (ViT), have been shown to outperform deep learning models constructed with convolutional neural networks (CNNs) on various vision t…

Audio ClassificationClassificationEmotion RecognitionKeyword Spotting+3