Visual Question Answering (VQA) 벤치마크
Visual Question Answering (VQA) on WHOOPS!
Exact Match
- 2023-03-13 — BLIP2 FlanT5-XXL (Fine-tuned): Exact Match 21.0
| Rank | Model | Exact Match | BEM | Extra Training Data | Paper | Code | Year |
|---|---|---|---|---|---|---|---|
| 1 | BLIP2 FlanT5-XXL (Fine-tuned) | 21 | 57 | ✓ | Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images | 2023 | |
| 2 | BLIP2 FlanT5-XL (Fine-tuned) | 20 | 55 | ✓ | Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images | 2023 | |
| 3 | BLIP2 FlanT5-XXL (Zero-shot) | 15 | 55 | Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images | 2023 | ||
| 4 | OFA Large | 8 | 38 | Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images | 2023 | ||
| 5 | BLIP Large | 6 | 39 | Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images | 2023 | ||
| 6 | BLIP2 FlanT5-XXL (Text-only FT) | 4 | 24 | ✓ | Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images | 2023 |