Offline Clustering Approach to Self-supervised Learning for Class-imbalanced Image Data
Class-imbalanced datasets are known to cause the problem of model being biased towards the majority classes. In this project, we set up two research questions: 1) when is the class-imbalance problem more prevalent in self-supervised pre-training? and 2) can offline clustering of feature representations help pre-training on class-imbalanced data? Our experiments investigate the former question by adjusting the degree of {\it class-imbalance} when training the baseline models, namely SimCLR and SimSiam on CIFAR-10 database. To answer the latter question, we train each expert model on each subset of the feature clusters. We then distill the knowledge of expert models into a single model, so that we will be able to compare the performance of this model to our baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
S3PT: Scene Semantics and Structure Guided Clustering to Boost Self-Supervised Pre-Training for Autonomous Driving
Recent self-supervised clustering-based pre-training techniques like DINO and Cribo have shown impressive results for downstream detection and segmentation tasks. However, real-world applications such as autonomous drivi…
3D Object DetectionAutonomous DrivingClusteringDiversity+4Improving ClusterGAN Using Self-Augmented Information Maximization of Disentangling Latent Spaces
Since their introduction in the last few years, conditional generative models have seen remarkable achievements. However, they often need the use of large amounts of labelled information. By using unsupervised conditiona…
ClusteringThe Hidden Uniform Cluster Prior in Self-Supervised Learning
A successful paradigm in representation learning is to perform self-supervised pretraining using tasks based on mini-batch statistics (e.g., SimCLR, VICReg, SwAV, MSN). We show that in the formulation of all these method…
ClusteringRepresentation LearningSelf-Supervised LearningDECAR: Deep Clustering for learning general-purpose Audio Representations
We introduce DECAR, a self-supervised pre-training approach for learning general-purpose audio representations. Our system is based on clustering: it utilizes an offline clustering step to provide target labels that act …
ClusteringDeep ClusteringSelf-Supervised LearningTransfer LearningScalable Exemplar-based Subspace Clustering on Class-Imbalanced Data
Subspace clustering methods based on expressing each data point as a linear combination of a few other data points (e.g., sparse subspace clustering) have become a popular tool for unsupervised learning due to their empi…
Clusteringimage-classificationImage Classification