Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?
The recent development of foundation models for time series data has generated considerable interest in using such models across a variety of applications. Although foundation models achieve state-of-the-art predictive performance, their calibration properties remain relatively underexplored, despite the fact that calibration can be critical for many practical applications. In this paper, we investigate the calibration-related properties of five recent time series foundation models and two competitive baselines. We perform a series of systematic evaluations assessing model calibration (i.e., over- or under-confidence), effects of varying prediction heads, and calibration under long-term autoregressive forecasting. We find that time series foundation models are consistently better calibrated than baseline models and tend not to be either systematically over- or under-confident, in contrast to the overconfidence often seen in other deep learning models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Multivariate Financial Forecasting using the Chronos Time Series Foundation Models
Using Chronos-2, an open-source time-series foundation model, we evaluate pretrained time-series models for economic and financial forecasting with an emphasis on whether multivariate (MV) inputs improve accuracy relativ…
Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning
Recently-developed time series foundation models for scientific machine learning exhibit emergent abilities to predict physical systems. These abilities include zero-shot forecasting, in which a model forecasts future st…
In-Context LearningTime SeriesTime Series ForecastingExploring Accuracy Law for Deep Time Series Forecasters: An Empirical Study
Deep time series forecasting has emerged as a rapidly growing field in recent years. Despite the exponential growth of community interests, progress on standard benchmarks is often limited to marginal improvements. A com…
Univariate Time Series ForecastingA decoder-only foundation model for time-series forecasting
Motivated by recent advances in large language models for Natural Language Processing (NLP), we design a time-series foundation model for forecasting whose out-of-the-box zero-shot performance on a variety of public data…
DecoderTime SeriesTime Series ForecastingBeyond Data Scarcity: A Frequency-Driven Framework for Zero-Shot Forecasting
Time series forecasting is critical in numerous real-world applications, requiring accurate predictions of future values based on observed patterns. While traditional forecasting techniques work well in in-domain scenari…
Few-Shot LearningSynthetic Data GenerationTime SeriesTime Series Forecasting