Probing Few-Shot Generalization with Attributes
Despite impressive progress in deep learning, generalizing far beyond the training distribution is an important open challenge. In this work, we consider few-shot classification, and aim to shed light on what makes some novel classes easier to learn than others, and what types of learned representations generalize better. To this end, we define a new paradigm in terms of attributes -- simple building blocks of which concepts are formed -- as a means of quantifying the degree of relatedness of different concepts. Our empirical analysis reveals that supervised learning generalizes poorly to new attributes, but a combination of self-supervised pretraining with supervised finetuning leads to stronger generalization. The benefit of self-supervised pretraining and supervised finetuning is further investigated through controlled experiments using random splits of the attribute space, and we find that predictability of test attributes provides an informative estimate of a model's generalization ability.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeFew-Shot LearningZero-Shot LearningSimilar Papers 제목 키워드 기반
Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition
Few-shot adaptation for Vision-Language Models (VLMs) presents a dilemma: balancing in-distribution accuracy with out-of-distribution generalization. Recent research has utilized low-level concepts such as visual attribu…
AttributeDecision MakingOut-of-Distribution Generalizationparameter-efficient fine-tuningSocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples
While vision-language models (VLMs) have achieved remarkable performance improvements recently, there is growing evidence that these models also posses harmful biases with respect to social attributes such as gender and …
counterfactualZero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study
Domain shift, caused by variations in imaging modalities and acquisition protocols, limits model generalization in medical image segmentation. While foundation models (FMs) trained on diverse large-scale data hold promis…
Domain GeneralizationImage SegmentationMedical Image SegmentationSegmentation+2Beyond Image-Text Matching: Verb Understanding in Multimodal Transformers Using Guided Masking
The dominant probing approaches rely on the zero-shot performance of image-text matching tasks to gain a finer-grained understanding of the representations learned by recent multimodal image-language transformer models. …
Image-text matchingText MatchingProbing Intersectional Biases in Vision-Language Models with Counterfactual Examples
While vision-language models (VLMs) have achieved remarkable performance improvements recently, there is growing evidence that these models also posses harmful biases with respect to social attributes such as gender and …
counterfactual