paper-with-me

Papers

Implicit Reasoning in Deep Time Series Forecasting

2024-09-17 · Willa Potosnak, Cristian Challu, Mononito Goswami, Michał Wiliński, Nina Żukowska, Artur Dubrawski

Recently, time series foundation models have shown promising zero-shot forecasting performance on time series from a wide range of domains. However, it remains unclear whether their success stems from a true understanding of temporal dynamics or simply from memorizing the training data. While implicit reasoning in language models has been studied, similar evaluations for time series models have been largely unexplored. This work takes an initial step toward assessing the reasoning abilities of deep time series forecasting models. We find that certain linear, MLP-based, and patch-based Transformer models generalize effectively in systematically orchestrated out-of-distribution scenarios, suggesting underexplored reasoning capabilities beyond simple pattern memorization.

📄 PDF Abstract BibTeX arXiv:2409.10840

Code (0)

등록된 구현이 없습니다.

Tasks

MemorizationTime SeriesTime Series Forecasting

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Multi-Head Attention 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Attention 설명 없음

Similar Papers 제목 키워드 기반

Probabilistic Time Series Forecasting with Implicit Quantile Networks

2021-07-08 · Adèle Gouttes, Kashif Rasul, Mateusz Koren, Johannes Stephan 외

Here, we propose a general method for probabilistic time series forecasting. We combine an autoregressive recurrent neural network to model temporal dynamics with Implicit Quantile Networks to learn a large class of dist…

Probabilistic Time Series ForecastingTime SeriesTime Series AnalysisTime Series Forecasting

Language Models Still Struggle to Zero-shot Reason about Time Series

2024-04-17 · Mike A. Merrill, Mingtian Tan, Vinayak Gupta, Tom Hartvigsen 외

Time series are critical for decision-making in fields like finance and healthcare. Their importance has driven a recent influx of works passing time series into language models, leading to non-trivial forecasting on som…

Language ModelingLanguage ModellingQuestion AnsweringTime Series

KairosAgent: Agentic Time Series Forecasting with Fused Semantic Reasoning

2026-05-28 · Kun Feng, Ziwei Shan, Yuchen Fang, Yiyang Tan 외 arxiv

Cross-domain multimodal time series forecasting is a challenging task, requiring models to integrate precise numerical comprehension, cross-domain semantic understanding, and effective multimodal fusion. Existing approac…

Time Series ForecastingReinforcement Learning

From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection

2024-09-26 · Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu 외

This paper introduces a novel approach that leverages Large Language Models (LLMs) and Generative Agents to enhance time series forecasting by reasoning across both text and time series data. With language as a medium, o…

Time SeriesTime Series Forecasting

Long-term Forecasting with TiDE: Time-series Dense Encoder

2023-04-17 · Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan Mathur 외

Recent work has shown that simple linear models can outperform several Transformer based approaches in long term time-series forecasting. Motivated by this, we propose a Multi-layer Perceptron (MLP) based encoder-decoder…

Anomaly DetectionDecoderTime SeriesTime Series Forecasting