paper-with-me

Chart Question Answering 벤치마크

Chart Question Answering on ChartQA

54개 결과 · ⬇ CSV · JSON

1:1 Accuracy

36.9 48 59.1 70.2 81.3 2022-03 2026-09 VisionTapas-OCR — 45.5 (2022-03-19) VisionTapas-OCR — 45.5 (2022-03-19) Pix2Struct-large — 58.6 (2022-10-07) Pix2Struct-base — 56.0 (2022-10-07) Pix2Struct-large — 58.6 (2022-10-07) Pix2Struct-base — 56.0 (2022-10-07) MatCha — 64.2 (2022-12-19) MatCha — 64.2 (2022-12-19) DePlot+FlanPaLM+Codex (PoT Self-Consistency) — 79.3 (2022-12-20) DePlot+Codex (PoT Self-Consistency) — 76.7 (2022-12-20) DePlot+FlanPaLM (Self-Consistency) — 70.5 (2022-12-20) DePlot+FlanPaLM (CoT) — 67.3 (2022-12-20) DePlot+GPT3 (Self-Consistency) — 42.3 (2022-12-20) DePlot+GPT3 (CoT) — 36.9 (2022-12-20) DePlot+FlanPaLM+Codex (PoT Self-Consistency) — 79.3 (2022-12-20) DePlot+Codex (PoT Self-Consistency) — 76.7 (2022-12-20) DePlot+FlanPaLM (Self-Consistency) — 70.5 (2022-12-20) DePlot+FlanPaLM (CoT) — 67.3 (2022-12-20) DePlot+GPT3 (Self-Consistency) — 42.3 (2022-12-20) DePlot+GPT3 (CoT) — 36.9 (2022-12-20) UniChart — 66.24 (2023-05-24) UniChart — 66.24 (2023-05-24) PaLI-X (Single-task FT w/ OCR) — 72.3 (2023-05-29) PaLI-X (Single-task FT) — 70.9 (2023-05-29) PaLI-X (Multi-task FT) — 70.6 (2023-05-29) PaLI-X (Single-task FT w/ OCR) — 72.3 (2023-05-29) PaLI-X (Single-task FT) — 70.9 (2023-05-29) PaLI-X (Multi-task FT) — 70.6 (2023-05-29) Qwen-VL-Chat — 66.3 (2023-08-24) Qwen-VL — 65.7 (2023-08-24) Qwen-VL-Chat — 66.3 (2023-08-24) Qwen-VL — 65.7 (2023-08-24) StructChart+GPT3.5 (STR ChartQA+SimChart9K) — 65.3 (2023-09-20) StructChart+GPT3.5 (STR) — 60.7 (2023-09-20) StructChart+GPT3.5 (STR ChartQA+SimChart9K) — 65.3 (2023-09-20) StructChart+GPT3.5 (STR) — 60.7 (2023-09-20) PaLI-3 — 70.0 (2023-10-13) PaLI-3 (w/ OCR) — 69.5 (2023-10-13) PaLI-3 — 70.0 (2023-10-13) PaLI-3 (w/ OCR) — 69.5 (2023-10-13) SMoLA-PaLI-X Specialist Model — 74.6 (2023-12-01) SMoLA-PaLI-X Generalist Model — 73.8 (2023-12-01) SMoLA-PaLI-X Specialist Model — 74.6 (2023-12-01) SMoLA-PaLI-X Generalist Model — 73.8 (2023-12-01) Gemini Ultra — 80.8 (2023-12-19) Gemini Ultra — 80.8 (2023-12-19) MatCha4096 + LaMenDa — 72.64 (2024-01-01) MatCha4096 + LaMenDa — 72.64 (2024-01-01) ScreenAI 5B (4.62 B params, w/ OCR) — 76.7 (2024-02-07) ScreenAI 5B (4.62 B params, w/ OCR) — 76.7 (2024-02-07) ChartPaLI-5B + PaLM 2-S — 81.3 (2024-03-19) ChartPaLI-5B — 77.3 (2024-03-19) ChartPaLI-5B + PaLM 2-S — 81.3 (2024-03-19) ChartPaLI-5B — 77.3 (2024-03-19) VisionTapas-OCR — 45.5 (2022-03-19) Pix2Struct-large — 58.6 (2022-10-07) MatCha — 64.2 (2022-12-19) DePlot+FlanPaLM+Codex (PoT Self-Consistency) — 79.3 (2022-12-20) Gemini Ultra — 80.8 (2023-12-19) ChartPaLI-5B + PaLM 2-S — 81.3 (2024-03-19)
RankModel 1:1 Accuracy Extra Training Data PaperCodeYear
21 MatCha 64.2 MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering huggingface/transformers 2022
22 StructChart+GPT3.5 (STR) 60.7 StructChart: On the Schema, Metric, and Augmentation for Visual Chart Understanding alpha-innovator/chartvlm · unimodal4reasoning/chartvlm · unimodal4reasoning/simchart9k 2023
23 Pix2Struct-large 58.6 Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding huggingface/transformers · google-research/pix2struct · yangyucheng000/University · +1 2022
24 Pix2Struct-base 56.0 Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding huggingface/transformers · google-research/pix2struct · yangyucheng000/University · +1 2022
25 VisionTapas-OCR 45.5 ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning vis-nlp/chartqa 2022
26 DePlot+GPT3 (Self-Consistency) 42.3 DePlot: One-shot visual language reasoning by plot-to-table translation huggingface/transformers 2022
27 DePlot+GPT3 (CoT) 36.9 DePlot: One-shot visual language reasoning by plot-to-table translation huggingface/transformers 2022
28 ChartPaLI-5B + PaLM 2-S 81.3 ✓ Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs 2024
29 Gemini Ultra 80.8 Gemini: A Family of Highly Capable Multimodal Models valdecy/pybibx 2023
30 DePlot+FlanPaLM+Codex (PoT Self-Consistency) 79.3 DePlot: One-shot visual language reasoning by plot-to-table translation huggingface/transformers 2022
31 ChartPaLI-5B 77.3 ✓ Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs 2024
32 DePlot+Codex (PoT Self-Consistency) 76.7 DePlot: One-shot visual language reasoning by plot-to-table translation huggingface/transformers 2022
32 ScreenAI 5B (4.62 B params, w/ OCR) 76.7 ✓ ScreenAI: A Vision-Language Model for UI and Infographics Understanding google-research-datasets/screen_qa · google-research-datasets/screen_annotation 2024
34 SMoLA-PaLI-X Specialist Model 74.6 ✓ Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts 2023
35 SMoLA-PaLI-X Generalist Model 73.8 ✓ Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts 2023
36 MatCha4096 + LaMenDa 72.64 ✓ Synthesize Step-by-Step: Tools Templates and LLMs as Data Generators for Reasoning-Based Chart VQA 2024
37 PaLI-X (Single-task FT w/ OCR) 72.3 ✓ PaLI-X: On Scaling up a Multilingual Vision and Language Model kyegomez/PALI · doc-doc/NExT-OE 2023
38 PaLI-X (Single-task FT) 70.9 ✓ PaLI-X: On Scaling up a Multilingual Vision and Language Model kyegomez/PALI · doc-doc/NExT-OE 2023
39 PaLI-X (Multi-task FT) 70.6 ✓ PaLI-X: On Scaling up a Multilingual Vision and Language Model kyegomez/PALI · doc-doc/NExT-OE 2023
40 DePlot+FlanPaLM (Self-Consistency) 70.5 DePlot: One-shot visual language reasoning by plot-to-table translation huggingface/transformers 2022
← 이전 21–40 / 54 다음 → 페이지당 10 20 50 100