Papers Zero-Shot Image Classification
“Zero-Shot Image Classification” 태그가 달린 논문 128편 · 필터 해제
Bayesian Test-Time Adaptation for Vision-Language Models
Test-time adaptation with pre-trained vision-language models, such as CLIP, aims to adapt the model to new, potentially out-of-distribution test data. Existing methods calculate the similarity between visual embedding an…
image-classificationImage ClassificationTest-time AdaptationZero-Shot Image ClassificationMADS: Multi-Attribute Document Supervision for Zero-Shot Image Classification
Zero-shot learning (ZSL) aims to train a model on seen classes and recognize unseen classes by knowledge transfer through shared auxiliary information. Recent studies reveal that documents from encyclopedias provide help…
Attributeimage-classificationImage ClassificationTransfer Learning+2MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations
Despite significant progress in Vision-Language Pre-training (VLP), current approaches predominantly emphasize feature extraction and cross-modal comprehension, with limited attention to generating or transforming visual…
image-classificationImage ClassificationImage GenerationImage-text matching+7Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion
Pre-trained multi-modal Vision-Language Models like CLIP are widely used off-the-shelf for a variety of applications. In this paper, we show that the common practice of individually exploiting the text or image encoders …
image-classificationImage ClassificationImage RetrievalRetrieval+2KPL: Training-Free Medical Knowledge Mining of Vision-Language Models
Visual Language Models such as CLIP excel in image recognition due to extensive image-text pre-training. However, applying the CLIP inference in zero-shot classification, particularly for medical image diagnosis, faces c…
Classificationimage-classificationImage ClassificationMedical Image Classification+3Retaining Knowledge and Enhancing Long-Text Representations in CLIP through Dual-Teacher Distillation
Contrastive language-image pretraining models such as CLIP have demonstrated remarkable performance in various text-image alignment tasks. However, the inherent 77-token input limitation and reliance on predominantly…
image-classificationImage ClassificationImage to textImage-to-Text Retrieval+3Post-hoc Probabilistic Vision-Language Models
Vision-language models (VLMs), such as CLIP and SigLIP, have found remarkable success in classification, retrieval, and generative tasks. For this, VLMs deterministically map images and text descriptions to a joint laten…
Active LearningUncertainty QuantificationZero-Shot Image ClassificationCLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance
Beyond the success of Contrastive Language-Image Pre-training (CLIP), recent trends mark a shift toward exploring the applicability of lightweight vision-language models for resource-constrained scenarios. These models o…
Contrastive Learningcross-modal alignmentCross-Modal RetrievalLinear evaluation+6TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives
Contrastive Language-Image Pretraining (CLIP) models maximize the mutual information between text and visual modalities to learn representations. This makes the nature of the training data a significant factor in the eff…
Diversityimage-classificationImage ClassificationImage Retrieval+2TaxaBind: A Unified Embedding Space for Ecological Applications
We present TaxaBind, a unified embedding space for characterizing any species of interest. TaxaBind is a multimodal embedding space across six modalities: ground-level images of species, geographic location, satellite im…
Audio ClassificationCross-Modal RetrievalZero-Shot Image ClassificationRetrieval-enriched zero-shot image classification in low-resource domains
Low-resource domains, characterized by scarce data and annotations, present significant challenges for language and visual understanding tasks, with the latter much under-explored in the literature. Recent advancements i…
image-classificationImage ClassificationRetrievalSynthetic Data Generation+1Multilingual Vision-Language Pre-training for the Remote Sensing Domain
Methods based on Contrastive Language-Image Pre-training (CLIP) are nowadays extensively used in support of vision-and-language tasks involving remote sensing data, such as cross-modal retrieval. The adaptation of CLIP t…
Cross-Modal Retrievalimage-classificationImage ClassificationImage-text Retrieval+4Altogether: Image Captioning via Re-aligning Alt-text
This paper focuses on creating synthetic data to improve the quality of image captions. Existing works typically have two shortcomings. First, they caption images from scratch, ignoring existing alt-text metadata, and se…
Image Captioningimage-classificationImage ClassificationImage Generation+3Open-vocabulary vs. Closed-set: Best Practice for Few-shot Object Detection Considering Text Describability
Open-vocabulary object detection (OVD), detecting specific classes of objects using only their linguistic descriptions (e.g., class names) without any image samples, has garnered significant attention. However, in real-w…
Few-Shot Object Detectionimage-classificationImage ClassificationObject+5Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge
Contrastive Language-Image Pretraining (CLIP) performs zero-shot image classification by mapping images and textual class representation into a shared embedding space, then retrieving the class closest to the image. This…
Classificationimage-classificationImage Classificationzero-shot-classification+2CSA: Data-efficient Mapping of Unimodal Features to Multimodal Features
Multimodal encoders like CLIP excel in tasks such as zero-shot image classification and cross-modal retrieval. However, they require excessive training data. We propose canonical similarity analysis (CSA), which uses two…
Cross-Modal RetrievalGPUimage-classificationImage Classification+1LoGra-Med: Long Context Multi-Graph Alignment for Medical Vision-Language Model
State-of-the-art medical multi-modal large language models (med-MLLM), like LLaVA-Med or BioMedGPT, leverage instruction-following data in pre-training. However, those models primarily focus on scaling the model size and…
image-classificationImage ClassificationInstruction FollowingLanguage Modeling+4CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling
In recent years, Contrastive Language-Image Pre-training (CLIP) has become a cornerstone in multimodal intelligence. However, recent studies have identified that the information loss in the CLIP encoding process is subst…
image-classificationImage ClassificationLarge Language ModelMixture-of-Experts+2DPA: Dual Prototypes Alignment for Unsupervised Adaptation of Vision-Language Models
Vision-language models (VLMs), e.g., CLIP, have shown remarkable potential in zero-shot image classification. However, adapting these models to new domains remains challenging, especially in unsupervised settings where l…
Domain Adaptationimage-classificationImage ClassificationPseudo Label+2Do Vision-Language Foundational models show Robust Visual Perception?
Recent advances in vision-language foundational models have enabled development of systems that can perform visual understanding and reasoning tasks. However, it is unclear if these models are robust to distribution shif…
image-classificationImage ClassificationZero-Shot Image Classification