A Survey on Contrastive Self-supervised Learning
Self-supervised learning has gained popularity because of its ability to avoid the cost of annotating large-scale datasets. It is capable of adopting self-defined pseudo labels as supervision and use the learned representations for several downstream tasks. Specifically, contrastive learning has recently become a dominant component in self-supervised learning methods for computer vision, natural language processing (NLP), and other domains. It aims at embedding augmented versions of the same sample close to each other while trying to push away embeddings from different samples. This paper provides an extensive review of self-supervised methods that follow the contrastive approach. The work explains commonly used pretext tasks in a contrastive learning setup, followed by different architectures that have been proposed so far. Next, we have a performance comparison of different methods for multiple downstream tasks such as image classification, object detection, and action recognition. Finally, we conclude with the limitations of the current methods and the need for further techniques and future directions to make substantial progress.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionContrastive Learningimage-classificationImage Classificationobject-detectionObject DetectionSelf-Supervised LearningSurveyMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Contrastive Self-supervised Learning in Recommender Systems: A Survey
Deep learning-based recommender systems have achieved remarkable success in recent years. However, these methods usually heavily rely on labeled data (i.e., user-item interactions), suffering from problems such as data s…
Recommendation SystemsSelf-Supervised LearningSurveySelf-supervised Learning: Generative or Contrastive
Deep supervised learning has achieved great success in the last decade. However, its deficiencies of dependence on manual labels and vulnerability to attacks have driven people to explore a better solution. As an alterna…
Graph LearningRepresentation LearningSelf-Supervised LearningSurveyHeterogeneous Contrastive Learning for Foundation Models and Beyond
In the era of big data and Artificial Intelligence, an emerging paradigm is to utilize contrastive self-supervised learning to model large-scale heterogeneous data. Many existing foundation models benefit from the genera…
Contrastive LearningSelf-Supervised LearningSurveyA Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives
Modern natural language processing (NLP) methods employ self-supervised pretraining objectives such as masked language modeling to boost the performance of various application tasks. These pretraining methods are frequen…
Contrastive LearningLanguage ModelingLanguage ModellingMasked Language Modeling+3Self-Supervised Representation Learning for Astronomical Images
Sky surveys are the largest data generators in astronomy, making automated tools for extracting meaningful scientific information an absolute necessity. We show that, without the need for labels, self-supervised learning…
AstronomyContrastive LearningMorphology classificationPhotometric Redshift Estimation+3