Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks
Recent work on few-shot learning \cite{tian2020rethinking} showed that quality of learned representations plays an important role in few-shot classification performance. On the other hand, the goal of self-supervised learning is to recover useful semantic information of the data without the use of class labels. In this work, we exploit the complementarity of both paradigms via a multi-task framework where we leverage recent self-supervised methods as auxiliary tasks. We found that combining multiple tasks is often beneficial, and that solving them simultaneously can be done efficiently. Our results suggest that self-supervised auxiliary tasks are effective data-dependent regularizers for representation learning. Our code is available at: \url{https://github.com/nathanielsimard/improving-fs-ssl}.
Code (1)
Tasks
Few-Shot LearningRepresentation LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Tailoring Self-Supervision for Supervised Learning
Recently, it is shown that deploying a proper self-supervision is a prospective way to enhance the performance of supervised learning. Yet, the benefits of self-supervision are not fully exploited as previous pretext tas…
Adversarial RobustnessData AugmentationImage Classificationimbalanced classification+2Multi-Pretext Attention Network for Few-shot Learning with Self-supervision
Few-shot learning is an interesting and challenging study, which enables machines to learn from few samples like humans. Existing studies rarely exploit auxiliary information from large amount of unlabeled data. Self-sup…
ClusteringFew-Shot LearningSelf-Supervised LearningInvestigating Self-Supervised Methods for Label-Efficient Learning
Vision transformers combined with self-supervised learning have enabled the development of models which scale across large datasets for several downstream tasks like classification, segmentation and detection. The low-sh…
ClassificationClusteringContrastive LearningMulti-class Classification+5Predicting What You Already Know Helps: Provable Self-Supervised Learning
Self-supervised representation learning solves auxiliary prediction tasks (known as pretext tasks) without requiring labeled data to learn useful semantic representations. These pretext tasks are created solely using the…
Representation LearningSelf-Supervised LearningImproving out-of-distribution generalization via multi-task self-supervised pretraining
Self-supervised feature representations have been shown to be useful for supervised classification, few-shot learning, and adversarial robustness. We show that features obtained using self-supervised learning are compara…
Adversarial RobustnessDomain GeneralizationFew-Shot LearningMulti-Task Learning+2