On the Internal Semantics of Time-Series Foundation Models

2025-11-19 · Atharva Pandey, Abhilash Neog, Gautam Jajoo arxiv

Time-series Foundation Models (TSFMs) have recently emerged as a universal paradigm for learning across diverse temporal domains. However, despite their empirical success, the internal mechanisms by which these models represent fundamental time-series concepts remain poorly understood. In this work, we undertake a systematic investigation of concept interpretability in TSFMs. Specifically, we examine: (i) which layers encode which concepts, (ii) whether concept parameters are linearly recoverable, (iii) how representations evolve in terms of concept disentanglement and abstraction across model depth, and (iv) how models process compositions of concepts. We systematically probe these questions using layer-wise analyses, linear recoverability tests, and representation similarity measures, providing a structured account of TSFM semantics. The resulting insights show that early layers mainly capture local, time-domain patterns (e.g., AR(1), level shifts, trends), while deeper layers encode dispersion and change-time signals, with spectral and warping factors remaining the hardest to recover linearly. In compositional settings, however, probe performance degrades, revealing interference between concepts. This highlights that while atomic concepts are reliably localized, composition remains a challenge, underscoring a key limitation in current TSFMs' ability to represent interacting temporal phenomena.

📄 PDF Abstract BibTeX arXiv:2511.15324

Code (0)

등록된 구현이 없습니다.

time2time: Causal Intervention in Hidden States to Simulate Rare Events in Time Series Foundation Models

2025-09-06 · Debdeep Sanyal, Aaryan Nagpal, Dhruv Kumar, Murari Mandal 외 arxiv

While transformer-based foundation models excel at forecasting routine patterns, two questions remain: do they internalize semantic concepts such as market regimes, or merely fit curves? And can their internal representa…

VLT: A Vision-Language-Time Series Multimodal Foundation Model for Industrial Intelligence

2026-07-16 · Haiteng Wang, Jingheng Yan, Xiaokang Wang, Lei Ren arxiv

Industrial time series serve as the foundation for Prognostics and Health Management (PHM) to ensure the reliability and safety of industrial equipment such as aero-engines. However, existing approaches are typically lim…

This Time is Different: An Observability Perspective on Time Series Foundation Models

2025-05-20 · Ben Cohen, Emaad Khwaja, Youssef Doubli, Salahidine Lemaachi 외

We introduce Toto, a time series forecasting foundation model with 151 million parameters. Toto uses a modern decoder-only architecture coupled with architectural innovations designed to account for specific challenges f…

Decoder Multivariate Time Series Forecasting Time Series Time Series Forecasting

Discrete Prototypical Memories for Federated Time Series Foundation Models

2026-04-06 · Liwei Deng, Qingxiang Liu, Xinhe Niu, Shengchao Chen 외 arxiv

Leveraging Large Language Models (LLMs) as federated learning (FL)-based time series foundation models offers a promising way to transfer the generalization capabilities of LLMs to time series data while preserving acces…

Federated Learning

MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models

2026-02-14 · Xiaoyun Yu, Li fan, Xiangfei Qiu, Nanqing Dong 외 arxiv

While Time Series Foundation Models (TSFMs) have demonstrated exceptional performance in generalized forecasting, their performance often degrades significantly when deployed in real-world vertical domains characterized …

Time Series Forecasting Domain Adaptation