Time-Series Contrastive Learning against False Negatives and Class Imbalance
As an exemplary self-supervised approach for representation learning, time-series contrastive learning has exhibited remarkable advancements in contemporary research. While recent contrastive learning strategies have focused on how to construct appropriate positives and negatives, in this study, we conduct theoretical analysis and find they have overlooked the fundamental issues: false negatives and class imbalance inherent in the InfoNCE loss-based framework. Therefore, we introduce a straightforward modification grounded in the SimCLR framework, universally adaptable to models engaged in the instance discrimination task. By constructing instance graphs to facilitate interactive learning among instances, we emulate supervised contrastive learning via the multiple-instances discrimination task, mitigating the harmful impact of false negatives. Moreover, leveraging the graph structure and few-labeled data, we perform semi-supervised consistency classification and enhance the representative ability of minority classes. We compared our method with the most popular time-series contrastive learning methods on four real-world time-series datasets and demonstrated our significant advantages in overall performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningRepresentation LearningTime SeriesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mitigating the Impact of False Negatives in Dense Retrieval with Contrastive Confidence Regularization
In open-domain Question Answering (QA), dense retrieval is crucial for finding relevant passages for answer generation. Typically, contrastive learning is used to train a retrieval model that maps passages and queries to…
Answer GenerationContrastive LearningOpen-Domain Question AnsweringQuestion Answering+2Incremental False Negative Detection for Contrastive Learning
Self-supervised learning has recently shown great potential in vision tasks through contrastive learning, which aims to discriminate each image, or instance, in the dataset. However, such instance-level learning ignores …
Contrastive LearningSelf-Supervised LearningE2USD: Efficient-yet-effective Unsupervised State Detection for Multivariate Time Series
Cyber-physical system sensors emit multivariate time series (MTS) that monitor physical system processes. Such time series generally capture unknown numbers of states, each with a different duration, that correspond to s…
Contrastive LearningTime SeriesFALCON: False-Negative Aware Learning of Contrastive Negatives in Vision-Language Pretraining
False negatives pose a critical challenge in vision-language pretraining (VLP) due to the many-to-many correspondence between images and texts in large-scale datasets. These false negatives introduce conflicting supervis…
cross-modal alignmentUnsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusion
Unsupervised/self-supervised time series representation learning is a challenging problem because of its complex dynamics and sparse annotations. Existing works mainly adopt the framework of contrastive learning with the…
Anomaly DetectionContrastive LearningRepresentation LearningTime Series+1