Object Counting 벤치마크
Object Counting on TallyQA-Complex
Accuracy
- 2018-10-29 — RCN: Accuracy 56.2
- 2020-04-24 — MoVie-ResNeXt: Accuracy 56.8
- 2023-12-01 — SMoLA-PaLI-X Specialist: Accuracy 77.1
| Rank | Model | Accuracy | RMSE | Extra Training Data | Paper | Code | Year |
|---|---|---|---|---|---|---|---|
| 1 | SMoLA-PaLI-X Specialist | 77.1 | – | ✓ | Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts | 2023 | |
| 2 | PaLI-X-VPD | 76.6 | – | Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models | 2023 | ||
| 3 | SMoLA-PaLI-X Generalist (0 shot) | 70.7 | – | ✓ | Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts | 2023 | |
| 4 | MoVie-ResNeXt | 56.8 | 1.43 | MoVie: Revisiting Modulated Convolutions for Visual Counting and Beyond | facebookresearch/mmf | 2020 | |
| 5 | RCN | 56.2 | 1.43 | TallyQA: Answering Complex Counting Questions | manoja328/tallyqacode | 2018 | |
| 6 | MoVie | 54.1 | 1.52 | MoVie: Revisiting Modulated Convolutions for Visual Counting and Beyond | facebookresearch/mmf | 2020 |