Object Counting 벤치마크
Object Counting on TallyQA-Simple
Accuracy
- 2018-10-29 — RCN: Accuracy 71.8
- 2020-04-24 — MoVie-ResNeXt: Accuracy 74.9
- 2023-12-01 — SMoLA-PaLI-X Specialist: Accuracy 86.3
| Rank | Model | Accuracy | RMSE | Extra Training Data | Paper | Code | Year |
|---|---|---|---|---|---|---|---|
| 1 | SMoLA-PaLI-X Specialist | 86.3 | – | ✓ | Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts | 2023 | |
| 2 | PaLI-X-VPD | 86.2 | – | Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models | 2023 | ||
| 3 | SMoLA-PaLI-X Generalist (0 shot) | 83.3 | – | ✓ | Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts | 2023 | |
| 4 | MoVie-ResNeXt | 74.9 | 1 | MoVie: Revisiting Modulated Convolutions for Visual Counting and Beyond | facebookresearch/mmf | 2020 | |
| 5 | RCN | 71.8 | 1.13 | TallyQA: Answering Complex Counting Questions | manoja328/tallyqacode | 2018 | |
| 6 | MoVie | 70.8 | 1.09 | MoVie: Revisiting Modulated Convolutions for Visual Counting and Beyond | facebookresearch/mmf | 2020 |