Class-relevant Patch Embedding Selection for Few-Shot Image Classification
Effective image classification hinges on discerning relevant features from both foreground and background ele- ments, with the foreground typically holding the critical informa- tion. While humans adeptly classify images with limited exposure, artificial neural networks often struggle with feature selection from rare samples. To address this challenge, we propose a novel method for selecting class-relevant patch embeddings. Our ap- proach involves splitting support and query images into patches, encoding them using a pre-trained Vision Transformer (ViT) to obtain class embeddings and patch embeddings, respectively. Subsequently, we filter patch embeddings using class embeddings to retain only the class-relevant ones. For each image, we calculate the similarity between class embedding and each patch embed- ding, sort the similarity sequence in descending order, and only retain top-ranked patch embeddings. By prioritizing similarity between the class embedding and patch embeddings, we select top-ranked patch embeddings to be fused with class embedding to form a comprehensive image representation, enhancing pattern recognition across instances. Our strategy effectively mitigates the impact of class-irrelevant patch embeddings, yielding improved performance in pre-trained models. Extensive experiments on popular few-shot classification benchmarks demonstrate the sim- plicity, efficacy, and computational efficiency of our approach, outperforming state-of-the-art baselines under both 5-shot and 1-shot scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational Efficiencyfeature selectionFew-Shot Image Classificationimage-classificationImage ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Class-Aware Patch Embedding Adaptation for Few-Shot Image Classification
"A picture is worth a thousand words", significantly beyond mere a categorization. Accompanied by that, many patches of the image could have completely irrelevant meanings with the categorization if they were indepen…
Few-Shot Image ClassificationFew-Shot Learningimage-classificationImage ClassificationLeveraging Vision-Language Embeddings for Zero-Shot Learning in Histopathology Images
Zero-shot learning holds tremendous potential for histopathology image analysis by enabling models to generalize to unseen classes without extensive labeled data. Recent advancements in vision-language models (VLMs) have…
Diagnosticimage-classificationImage Classificationzero-shot-classification+1Stochastic-based Patch Filtering for Few-Shot Learning
Food images present unique challenges for few-shot learning models due to their visual complexity and variability. For instance, a pasta dish might appear with various garnishes on different plates and in diverse lightin…
Few-Shot LearningSoft Head Selection for Injecting ICL-Derived Task Embeddings
Large language models (LLMs) are commonly adapted to downstream tasks using parameter-efficient fine-tuning (PEFT) or in-context learning (ICL). Recently, ICL-driven embedding-based adaptation has been proposed as a dist…
parameter-efficient fine-tuningNatural Language UnderstandingA Two-Fold Patch Selection Approach for Improved 360-Degree Image Quality Assessment
This article presents a novel approach to improving the accuracy of 360-degree perceptual image quality assessment (IQA) through a two-fold patch selection process. Our methodology combines visual patch selection with em…
Image Quality Assessment