Meta-Adapter: An Online Few-shot Learner for Vision-Language Model
The contrastive vision-language pre-training, known as CLIP, demonstrates remarkable potential in perceiving open-world visual concepts, enabling effective zero-shot image recognition. Nevertheless, few-shot learning methods based on CLIP typically require offline fine-tuning of the parameters on few-shot samples, resulting in longer inference time and the risk of over-fitting in certain domains. To tackle these challenges, we propose the Meta-Adapter, a lightweight residual-style adapter, to refine the CLIP features guided by the few-shot samples in an online manner. With a few training samples, our method can enable effective few-shot learning capabilities and generalize to unseen data or tasks without additional fine-tuning, achieving competitive performance and high efficiency. Without bells and whistles, our approach outperforms the state-of-the-art online few-shot learning method by an average of 3.6\% on eight image classification datasets with higher inference speed. Furthermore, our model is simple and flexible, serving as a plug-and-play module directly applicable to downstream tasks. Without further fine-tuning, Meta-Adapter obtains notable performance improvements in open-vocabulary object detection and segmentation tasks.
Code (1)
Tasks
Few-Shot Learningimage-classificationImage ClassificationLanguage ModelingLanguage Modellingobject-detectionObject DetectionOpen-vocabulary object detectionOpen Vocabulary Object DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model
Contrastive vision-language models excel in zero-shot image recognition but face challenges in few-shot scenarios due to computationally intensive offline fine-tuning using prompt learning, which risks overfitting. To ov…
Few-Shot LearningMeta-Adapters: Parameter Efficient Few-shot Fine-tuning through Meta-Learning
Consistent improvements in the representational capacity of large pre-trained transformers has made it increasingly viable to serve these models as shared priors that can be fine-tuned on a large number of downstream tas…
Few-Shot LearningMeta-Learningparameter-efficient fine-tuningTransfer LearningMeta-Adapter: Parameter Efficient Few-Shot Learning through Meta-Learning
With consistent improvements in the representational capacity of large pre-trained transformers, it has become increasingly viable to serve these models as shared backbones that enable modeling a large number of tasks si…
Few-Shot LearningMeta-Learningparameter-efficient fine-tuningTransfer LearningPrior-Knowledge and Attention-based Meta-Learning for Few-Shot Learning
Recently, meta-learning has been shown as a promising way to solve few-shot learning. In this paper, inspired by the human cognition process which utilizes both prior-knowledge and vision attention in learning new knowle…
Few-Shot LearningMeta-LearningFew-Shot Learning with a Strong Teacher
Few-shot learning (FSL) aims to generate a classifier using limited labeled examples. Many existing works take the meta-learning approach, constructing a few-shot learner that can learn from few-shot examples to generate…
Few-Shot LearningMeta-Learning