General Time-series Model for Universal Knowledge Representation of Multivariate Time-Series data
Universal knowledge representation is a central problem for multivariate time series(MTS) foundation models and yet remains open. This paper investigates this problem from the first principle and it makes four folds of contributions. First, a new empirical finding is revealed: time series with different time granularities (or corresponding frequency resolutions) exhibit distinct joint distributions in the frequency domain. This implies a crucial aspect of learning universal knowledge, one that has been overlooked by previous studies. Second, a novel Fourier knowledge attention mechanism is proposed to enable learning time granularity-aware representations from both the temporal and frequency domains. Third, an autoregressive blank infilling pre-training framework is incorporated to time series analysis for the first time, leading to a generative tasks agnostic pre-training strategy. To this end, we develop the General Time-series Model (GTM), a unified MTS foundation model that addresses the limitation of contemporary time series models, which often require token, pre-training, or model-level customizations for downstream tasks adaption. Fourth, extensive experiments show that GTM outperforms state-of-the-art (SOTA) methods across all generative tasks, including long-term forecasting, anomaly detection, and imputation.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionImputationTime SeriesTime Series AnalysisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Distilling Universal and Joint Knowledge for Cross-Domain Model Compression on Time Series Data
For many real-world time series tasks, the computational complexity of prevalent deep leaning models often hinders the deployment on resource-limited environments (e.g., smartphones). Moreover, due to the inevitable doma…
Knowledge DistillationModel CompressionTime SeriesTransfer LearningPath Signatures on Lie Groups
Path signatures are powerful nonparametric tools for time series analysis, shown to form a universal and characteristic feature map for Euclidean valued time series data. We lift the theory of path signatures to the sett…
Action RecognitionTemporal Action LocalizationTime SeriesTime Series AnalysisAre Synthetic Time-series Data Really not as Good as Real Data?
Time-series data presents limitations stemming from data quality issues, bias and vulnerabilities, and generalization problem. Integrating universal data synthesis methods holds promise in improving generalization. Howev…
Representation LearningTime SeriesAurora: Towards Universal Generative Multimodal Time Series Forecasting
Cross-domain generalization is very important in Time Series Forecasting because similar historical information may lead to distinct future trends due to the domain-specific characteristics. Recent works focus on buildin…
Time Series ForecastingDomain GeneralizationTimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis
Time series analysis plays a critical role in numerous applications, supporting tasks such as forecasting, classification, anomaly detection, and imputation. In this work, we present the time series pattern machine (TSPM…
Anomaly DetectionImputationTime SeriesTime Series Analysis