Zero-Shot Cross-Modal Retrieval on Flickr30k

22개 결과 · ⬇ CSV · JSON

Image-to-text R@1

Rank	Model	Image-to-text R@1	Image-to-text R@5	Image-to-text R@10	Text-to-image R@1	Text-to-image R@5	Text-to-image R@10	Extra Training Data	Paper	Code	Year
1	InternVL-G	95.7	99.7	99.9	85.0	97.0	98.6		InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks	opengvlab/internvl · opengvlab/internvl-mmdetseg	2023
2	BEiT-3	94.9	99.9	100.0	81.5	95.6	97.8		Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks	microsoft/unilm · lyan62/data-curation	2022
3	InternVL-C	94.7	99.6	99.9	81.7	96.0	98.2		InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks	opengvlab/internvl · opengvlab/internvl-mmdetseg	2023
4	COSMOS ViT-B/16	92.9	99.4	99.9	80.3	95.3	97.6		COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training	ExplainableML/cosmos	2024
5	CoCa	92.5	99.5	99.9	80.4	95.7	97.7		CoCa: Contrastive Captioners are Image-Text Foundation Models	mlfoundations/open_clip · facebookresearch/multimodal · lucidrains/CoCa-pytorch · +3	2022
6	RO-ViT	92.1	99.4	99.7	80.7	96.1	97.7		Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers	google-research/google-research · mcahny/rovit	2023
7	M2-Encoder	91.2	99.2	99.6	92.2	99.5	99.7	✓	M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining	alipay/Ant-Multi-Modal-Framework	2024
7	ERNIE-ViL 2.0	91.2	99.1	99.8	77.4	93.8	96.4		ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training	PaddlePaddle/ERNIE	2022
9	Florence	90.9	99.1	-	76.7	93.6	-		Florence: A New Foundation Model for Computer Vision	microsoft/unicl · MindCode-4/code-3	2021
10	ALBEF	90.5	98.8	99.7	76.8	93.7	96.7		Align before Fuse: Vision and Language Representation Learning with Momentum Distillation	salesforce/lavis · salesforce/ALBEF · facebookresearch/multimodal · +3	2021
11	COSMOS ViT-B/32	89.9	98.8	99.3	76.1	92.8	96.2		COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training	ExplainableML/cosmos	2024
12	Flamingo	89.3	98.8	99.7	79.5	95.3	97.9		Flamingo: a Visual Language Model for Few-Shot Learning	mlfoundations/open_flamingo · lucidrains/flamingo-pytorch · unispac/visual-adversarial-examples-jailbreak-large-language-models · +2	2022
13	VK-OOD	89.0	99.2	99.8	77.2	94.3	98.2		Implicit Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis	ellenzhuwang/implicit_vkood	2023
14	ALIGN	88.6	98.7	99.7	75.7	93.8	96.8		Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision	facebookresearch/metaclip · kakaobrain/coyo-dataset · MicPie/clasp · +2	2021
15	CLIP	88.0	98.7	99.4	68.7	90.6	95.2		Learning Transferable Visual Models From Natural Language Supervision	openai/CLIP · mlfoundations/open_clip · towhee-io/towhee · +79	2021
16	PTP-BLIP (14M)	87.1	98.4	99.3	73.1	91.0	94.8		Position-guided Text Prompt for Vision-Language Pre-training	sail-sg/ptp	2022
17	AltCLIP	86	98	99.1	72.5	91.6	95.4		AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities	flagai-open/flagai · pwc-1/Paper-8	2022
18	UNITER	80.7	95.7	98.0	66.2	88.4	92.9		UNITER: UNiversal Image-TExt Representation Learning	ChenRocks/UNITER · YIKUAN8/Transformers-VQA · necla-ml/SNLI-VE · +4	2019
19	ViLT-B/32	73.2	93.6	96.5	55	82.5	89.8		ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision	huggingface/transformers · dandelin/vilt · glamor-usc/climb · +3	2021
20	ImageBERT	70.7	90.2	94.0	54.3	79.6	87.5		ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data		2020