DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning
Self-supervised learning (SSL) has gained remarkable success, for which contrastive learning (CL) plays a key role. However, the recent development of new non-CL frameworks has achieved comparable or better performance with high improvement potential, prompting researchers to enhance these frameworks further. Assimilating CL into non-CL frameworks has been thought to be beneficial, but empirical evidence indicates no visible improvements. In view of that, this paper proposes a strategy of performing CL along the dimensional direction instead of along the batch direction as done in conventional contrastive learning, named Dimensional Contrastive Learning (DimCL). DimCL aims to enhance the feature diversity, and it can serve as a regularizer to prior SSL frameworks. DimCL has been found to be effective, and the hardness-aware property is identified as a critical reason for its success. Extensive experimental results reveal that assimilating DimCL into SSL frameworks leads to performance improvement by a non-trivial margin on various datasets and backbone architectures.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningDiversitySelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Supervised Stochastic Neighbor Embedding Using Contrastive Learning
Stochastic neighbor embedding (SNE) methods $t$-SNE, UMAP are two most popular dimensionality reduction methods for data visualization. Contrastive learning, especially self-supervised contrastive learning (SSCL), has sh…
Contrastive LearningData VisualizationDimensionality ReductionUnderstanding Dimensional Collapse in Contrastive Self-supervised Learning
Self-supervised visual representation learning aims to learn useful representations without relying on human annotations. Joint embedding approach bases on maximizing the agreement between embedding vectors from differen…
Contrastive LearningLearning TheoryRepresentation LearningSelf-Supervised LearningUnified Framework for Feature Extraction based on Contrastive Learning
Feature extraction is an efficient approach for alleviating the issue of dimensionality in high-dimensional data. As a popular self-supervised learning method, contrastive learning has recently garnered considerable atte…
Contrastive LearningGraph EmbeddingSelf-Supervised LearningMetaMask: Revisiting Dimensional Confounder for Self-Supervised Learning
As a successful approach to self-supervised learning, contrastive learning aims to learn invariant information shared among distortions of the input sample. While contrastive learning has yielded continuous advancements …
Contrastive LearningMeta-LearningSelf-Supervised LearningFrom Pretext to Purpose: Batch-Adaptive Self-Supervised Learning
In recent years, self-supervised contrastive learning has emerged as a distinguished paradigm in the artificial intelligence landscape. It facilitates unsupervised feature learning through contrastive delineations at the…
Contrastive LearningDimensionality ReductionSelf-Supervised Learning