Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Mask
Time Series Representation Learning (TSRL) focuses on generating informative representations for various Time Series (TS) modeling tasks. Traditional Self-Supervised Learning (SSL) methods in TSRL fall into four main categories: reconstructive, adversarial, contrastive, and predictive, each with a common challenge of sensitivity to noise and intricate data nuances. Recently, diffusion-based methods have shown advanced generative capabilities. However, they primarily target specific application scenarios like imputation and forecasting, leaving a gap in leveraging diffusion models for generic TSRL. Our work, Time Series Diffusion Embedding (TSDE), bridges this gap as the first diffusion-based SSL TSRL approach. TSDE segments TS data into observed and masked parts using an Imputation-Interpolation-Forecasting (IIF) mask. It applies a trainable embedding function, featuring dual-orthogonal Transformer encoders with a crossover mechanism, to the observed part. We train a reverse diffusion process conditioned on the embeddings, designed to predict noise added to the masked part. Extensive experiments demonstrate TSDE's superiority in imputation, interpolation, forecasting, anomaly detection, classification, and clustering. We also conduct an ablation study, present embedding visualizations, and compare inference speed, further substantiating TSDE's efficiency and validity in learning representations of TS data.
Code (2)
Tasks
Anomaly DetectionImputationRepresentation LearningSelf-Supervised LearningTime SeriesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Diffusion Auto-regressive Transformer for Effective Self-supervised Time Series Forecasting
Self-supervised learning has become a popular and effective approach for enhancing time series forecasting, enabling models to learn universal representations from unlabeled data. However, effectively capturing both the …
DecoderDenoisingSelf-Supervised LearningTime Series+1Series2Vec: Similarity-based Self-supervised Representation Learning for Time Series Classification
We argue that time series analysis is fundamentally different in nature to either vision or natural language processing with respect to the forms of meaningful self-supervised learning tasks that can be defined. Motivate…
Data AugmentationRepresentation LearningSelf-Supervised LearningTime Series+2DiffStyleTS: Diffusion Model for Style Transfer in Time Series
Style transfer combines the content of one signal with the style of another. It supports applications such as data augmentation and scenario simulation, helping machine learning models generalize in data-scarce domains. …
Anomaly DetectionData AugmentationStyle TransferTS-Diffusion: Generating Highly Complex Time Series with Diffusion Models
While current generative models have achieved promising performances in time-series synthesis, they either make strong assumptions on the data format (e.g., regularities) or rely on pre-processing approaches (e.g., inter…
DecoderMissing ValuesTime SeriesSelf-Supervised Time Series Representation Learning by Inter-Intra Relational Reasoning
Self-supervised learning achieves superior performance in many domains by extracting useful representations from the unlabeled data. However, most of traditional self-supervised methods mainly focus on exploring the inte…
RelationRelational ReasoningRepresentation LearningSelf-Supervised Learning+3