Are Language Models Actually Useful for Time Series Forecasting?

2024-06-22 · Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff, Thomas Hartvigsen

Large language models (LLMs) are being applied to time series forecasting. But are language models actually useful for time series? In a series of ablation studies on three recent and popular LLM-based time series forecasting methods, we find that removing the LLM component or replacing it with a basic attention layer does not degrade forecasting performance -- in most cases, the results even improve! We also find that despite their significant computational cost, pretrained LLMs do no better than models trained from scratch, do not represent the sequential dependencies in time series, and do not assist in few-shot settings. Additionally, we explore time series encoders and find that patching and attention structures perform similarly to LLM-based forecasters.

📄 PDF Abstract BibTeX arXiv:2406.16964

Code (3)

bennytmt/llmsfortimeseries 공식 구현 pytorch

bennytmt/ts_models 공식 구현 pytorch

thuml/AutoTimes pytorch

Tasks

Time SeriesTime Series Forecasting

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

Patching Activation patching studies the model's computation by altering its latent representations, the token embeddings in transformer-based language models, during the inference process

From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?

2025-05-29 · Ziming Zhao, ChengAo Shen, Hanghang Tong, Dongjin Song 외

Transformer-based models have gained increasing attention in time series research, driving interest in Large Language Models (LLMs) and foundation models for time series analysis. As the field moves toward multi-modality…

Time Series Time Series Analysis Time Series Classification

Few-shot Learning for Time-series Forecasting

2020-09-30 · Tomoharu Iwata, Atsutoshi Kumagai

Time-series forecasting is important for many applications. Forecasting models are usually trained using time-series data in a specific target task. However, sufficient data in the target task might be unavailable, which…

Few-Shot Learning Time Series Time Series Analysis Time Series Forecasting

FinVerse: Financial Time-Series Benchmark

2026-08-04 · Jaehoon Lee, Jun Seo, Seunghan Lee, Tae Yoon Lim 외 arxiv

As time-series foundation models have emerged, the need for benchmarks that can evaluate their forecasting ability in meaningful ways has become increasingly important. Existing time-series forecasting benchmarks provide…

Decision Making

Trio: Learning Time-Series Forecasting with Temporal-Spatial-Sample Attention and Structural Causal Priors

2026-06-05 · Tao Chen, Yexu Zhou, Zhi Gong, Hengwei He 외 arxiv

Multivariate time-series forecasting requires models to reason over temporal dynamics, cross-variable dependencies, and historical input-output correspondences. Recent Prior-Data Fitted Networks (PFNs) suggest that synth…

Are Large Language Models Useful for Time Series Data Analysis?

2024-12-16 · Francis Tang, Ying Ding

Time series data plays a critical role across diverse domains such as healthcare, energy, and finance, where tasks like classification, anomaly detection, and forecasting are essential for informed decision-making. Recen…

Anomaly Detection Decision Making Time Series Time Series Analysis