An empirical study of pretrained representations for few-shot classification
Recent algorithms with state-of-the-art few-shot classification results start their procedure by computing data features output by a large pretrained model. In this paper we systematically investigate which models provide the best representations for a few-shot image classification task when pretrained on the Imagenet dataset. We test their representations when used as the starting point for different few-shot classification algorithms. We observe that models trained on a supervised classification task have higher performance than models trained in an unsupervised manner even when transferred to out-of-distribution datasets. Models trained with adversarial robustness transfer better, while having slightly lower accuracy than supervised models.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial RobustnessClassificationFew-Shot Image ClassificationGeneral Classificationimage-classificationImage ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On the Transferability of Large-Scale Self-Supervision to Few-Shot Audio Classification
In recent years, self-supervised learning has excelled for its capacity to learn robust feature representations from unlabelled data. Networks pretrained through self-supervision serve as effective feature extractors for…
Audio ClassificationFew-Shot Audio ClassificationFew-Shot LearningSelf-Supervised LearningZero-Shot Robustification of Zero-Shot Models
Zero-shot inference is a powerful paradigm that enables the use of large pretrained models for downstream classification tasks without further training. However, these models are vulnerable to inherited biases that can i…
Self-Supervised Learning for Few-Shot Bird Sound Classification
Self-supervised learning (SSL) in audio holds significant potential across various domains, particularly in situations where abundant, unlabeled data is readily available at no cost. This is pertinent in bioacoustics, wh…
ClassificationFew-Shot LearningSelf-Supervised LearningSound ClassificationFew-shot Classification as Multi-instance Verification: Effective Backbone-agnostic Transfer across Domains
We investigate cross-domain few-shot learning under the constraint that fine-tuning of backbones (i.e., feature extractors) is impossible or infeasible -- a scenario that is increasingly common in practical use cases. Ha…
cross-domain few-shot learningImage ClassificationDomain AdaptationDo Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
Prompting is a common approach for leveraging LMs in zero-shot settings. However, the underlying mechanisms that enable LMs to perform diverse tasks without task-specific supervision remain poorly understood. Studying th…