Self-Supervised Learning for Time Series: Contrastive or Generative?
Self-supervised learning (SSL) has recently emerged as a powerful approach to learning representations from large-scale unlabeled data, showing promising results in time series analysis. The self-supervised representation learning can be categorized into two mainstream: contrastive and generative. In this paper, we will present a comprehensive comparative study between contrastive and generative methods in time series. We first introduce the basic frameworks for contrastive and generative SSL, respectively, and discuss how to obtain the supervision signal that guides the model optimization. We then implement classical algorithms (SimCLR vs. MAE) for each type and conduct a comparative analysis in fair settings. Our results provide insights into the strengths and weaknesses of each approach and offer practical recommendations for choosing suitable SSL methods. We also discuss the implications of our findings for the broader field of representation learning and propose future research directions. All the code and data are released at \url{https://github.com/DL4mHealth/SSL_Comparison}.
Code (1)
Tasks
Model OptimizationRepresentation LearningSelf-Supervised LearningTime SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
A Unified Contrastive-Generative Framework for Time Series Classification
Self-supervised learning (SSL) for multivariate time series mainly includes two paradigms: contrastive methods that excel at instance discrimination and generative approaches that model data distributions. While effectiv…
Time Series ClassificationSelf-Supervised LearningContrastive LearningTi-MAE: Self-Supervised Masked Time Series Autoencoders
Multivariate Time Series forecasting has been an increasingly popular topic in various applications and scenarios. Recently, contrastive learning and Transformer-based models have achieved good performance in many long-t…
Contrastive LearningMultivariate Time Series ForecastingRepresentation LearningTime Series+2Unsupervised Driving Event Discovery Based on Vehicle CAN-data
The data collected from a vehicle's Controller Area Network (CAN) can quickly exceed human analysis or annotation capabilities when considering fleets of vehicles, which stresses the importance of unsupervised machine le…
Contrastive LearningSelf-Supervised LearningTime SeriesTime Series AnalysisCARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection
One main challenge in time series anomaly detection (TSAD) is the lack of labelled data in many real-life scenarios. Most of the existing anomaly detection methods focus on learning the normal behaviour of unlabelled tim…
Anomaly DetectionContrastive LearningRepresentation LearningTime Series+1Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training a…
Anomaly DetectionSelf-Supervised LearningTime SeriesTime Series Analysis+1