Multi-Task Self-Supervised Time-Series Representation Learning
Time-series representation learning can extract representations from data with temporal dynamics and sparse labels. When labeled data are sparse but unlabeled data are abundant, contrastive learning, i.e., a framework to learn a latent space where similar samples are close to each other while dissimilar ones are far from each other, has shown outstanding performance. This strategy can encourage varied consistency of time-series representations depending on the positive pair selection and contrastive loss. We propose a new time-series representation learning method by combining the advantages of self-supervised tasks related to contextual, temporal, and transformation consistency. It allows the network to learn general representations for various downstream tasks and domains. Specifically, we first adopt data preprocessing to generate positive and negative pairs for each self-supervised task. The model then performs contextual, temporal, and transformation contrastive learning and is optimized jointly using their contrastive losses. We further investigate an uncertainty weighting approach to enable effective multi-task learning by considering the contribution of each consistency. We evaluate the proposed framework on three downstream tasks: time-series classification, forecasting, and anomaly detection. Experimental results show that our method not only outperforms the benchmark models on these downstream tasks, but also shows efficiency in cross-domain transfer learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionContrastive LearningMulti-Task LearningRepresentation LearningTime SeriesTime Series AnalysisTime Series ClassificationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Series2Vec: Similarity-based Self-supervised Representation Learning for Time Series Classification
We argue that time series analysis is fundamentally different in nature to either vision or natural language processing with respect to the forms of meaningful self-supervised learning tasks that can be defined. Motivate…
Data AugmentationRepresentation LearningSelf-Supervised LearningTime Series+2MTSMAE: Masked Autoencoders for Multivariate Time-Series Forecasting
Large-scale self-supervised pre-training Transformer architecture have significantly boosted the performance for various tasks in natural language processing (NLP) and computer vision (CV). However, there is a lack of re…
Multivariate Time Series ForecastingSelf-Supervised LearningTime SeriesTime Series Analysis+1Self-Supervised Time Series Representation Learning by Inter-Intra Relational Reasoning
Self-supervised learning achieves superior performance in many domains by extracting useful representations from the unlabeled data. However, most of traditional self-supervised methods mainly focus on exploring the inte…
RelationRelational ReasoningRepresentation LearningSelf-Supervised Learning+3LiPCoT: Linear Predictive Coding based Tokenizer for Self-supervised Learning of Time Series Data via Language Models
Language models have achieved remarkable success in various natural language processing tasks. However, their application to time series data, a crucial component in many domains, remains limited. This paper proposes LiP…
EEGSelf-Supervised LearningTime SeriesDiffusion Auto-regressive Transformer for Effective Self-supervised Time Series Forecasting
Self-supervised learning has become a popular and effective approach for enhancing time series forecasting, enabling models to learn universal representations from unlabeled data. However, effectively capturing both the …
DecoderDenoisingSelf-Supervised LearningTime Series+1