paper-with-me

Math Word Problem Solving 벤치마크

Math Word Problem Solving on MAWPS

100개 결과 · ⬇ CSV · JSON

Accuracy (%)

2.76 25.99 49.23 72.47 95.7 2020-07 2026-11 Graph2Tree — 83.7 (2020-07-01) Graph2Tree — 83.7 (2020-07-01) Graph2Tree — 83.7 (2020-07-01) Graph2Tree — 83.7 (2020-07-01) GEO — 85.1 (2020-12-01) GEO — 85.1 (2020-12-01) GEO — 85.1 (2020-12-01) GEO — 85.1 (2020-12-01) Graph2Tree with RoBERTa — 88.7 (2021-03-12) GTS with RoBERTa — 88.5 (2021-03-12) Graph2Tree with RoBERTa — 88.7 (2021-03-12) GTS with RoBERTa — 88.5 (2021-03-12) Graph2Tree with RoBERTa — 88.7 (2021-03-12) GTS with RoBERTa — 88.5 (2021-03-12) Graph2Tree with RoBERTa — 88.7 (2021-03-12) GTS with RoBERTa — 88.5 (2021-03-12) Roberta-DeductReasoner — 92.0 (2022-03-19) Roberta-DeductReasoner — 92.0 (2022-03-19) Roberta-DeductReasoner — 92.0 (2022-03-19) Roberta-DeductReasoner — 92.0 (2022-03-19) Multi-view — 92.3 (2022-10-21) Multi-view — 92.3 (2022-10-21) Multi-view — 92.3 (2022-10-21) Multi-view — 92.3 (2022-10-21) MsAT-DeductReasoner — 94.3 (2023-06-02) MsAT-DeductReasoner — 94.3 (2023-06-02) MsAT-DeductReasoner — 94.3 (2023-06-02) MsAT-DeductReasoner — 94.3 (2023-06-02) DeBERTa (PM + VM) — 91.0 (2023-06-24) GPT-3.5 turbo (175B) — 80.3 (2023-06-24) GPT-J — 9.9 (2023-06-24) GPT-3 text-curie-001 (13B) — 4.09 (2023-06-24) GPT-3 text-babbage-001 (6.7B) — 2.76 (2023-06-24) DeBERTa (PM + VM) — 91.0 (2023-06-24) GPT-3.5 turbo (175B) — 80.3 (2023-06-24) GPT-J — 9.9 (2023-06-24) GPT-3 text-curie-001 (13B) — 4.09 (2023-06-24) GPT-3 text-babbage-001 (6.7B) — 2.76 (2023-06-24) DeBERTa (PM + VM) — 91.0 (2023-06-24) GPT-3.5 turbo (175B) — 80.3 (2023-06-24) GPT-J — 9.9 (2023-06-24) GPT-3 text-curie-001 (13B) — 4.09 (2023-06-24) GPT-3 text-babbage-001 (6.7B) — 2.76 (2023-06-24) DeBERTa (PM + VM) — 91.0 (2023-06-24) GPT-3.5 turbo (175B) — 80.3 (2023-06-24) GPT-J — 9.9 (2023-06-24) GPT-3 text-curie-001 (13B) — 4.09 (2023-06-24) GPT-3 text-babbage-001 (6.7B) — 2.76 (2023-06-24) LLaMA 2-Chat — 82.4 (2023-07-18) LLaMA 2-Chat — 82.4 (2023-07-18) LLaMA 2-Chat — 82.4 (2023-07-18) LLaMA 2-Chat — 82.4 (2023-07-18) Exp-Tree — 92.3 (2023-10-14) Exp-Tree — 92.3 (2023-10-14) Exp-Tree — 92.3 (2023-10-14) Exp-Tree — 92.3 (2023-10-14) ATHENA (roberta-large) — 93.0 (2023-11-02) ATHENA (roberta-base) — 92.2 (2023-11-02) ATHENA (roberta-large) — 93.0 (2023-11-02) ATHENA (roberta-base) — 92.2 (2023-11-02) ATHENA (roberta-large) — 93.0 (2023-11-02) ATHENA (roberta-base) — 92.2 (2023-11-02) ATHENA (roberta-large) — 93.0 (2023-11-02) ATHENA (roberta-base) — 92.2 (2023-11-02) OpenMath-CodeLlama-70B (w/ code) — 95.7 (2024-02-15) OpenMath-CodeLlama-70B (w/ code) — 95.7 (2024-02-15) OpenMath-CodeLlama-70B (w/ code) — 95.7 (2024-02-15) OpenMath-CodeLlama-70B (w/ code) — 95.7 (2024-02-15) Graph2Tree — 83.7 (2020-07-01) GEO — 85.1 (2020-12-01) Graph2Tree with RoBERTa — 88.7 (2021-03-12) Roberta-DeductReasoner — 92.0 (2022-03-19) Multi-view — 92.3 (2022-10-21) MsAT-DeductReasoner — 94.3 (2023-06-02) OpenMath-CodeLlama-70B (w/ code) — 95.7 (2024-02-15)
RankModel Accuracy (%) Extra Training Data PaperCodeYear
1 OpenMath-CodeLlama-70B (w/ code) 95.7 ✓ OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset kipok/nemo-skills 2024
2 MsAT-DeductReasoner 94.3 Learning Multi-Step Reasoning by Solving Arithmetic Tasks TianduoWang/MsAT 2023
3 ATHENA (roberta-large) 93 ATHENA: Mathematical Reasoning with Thought Expansion the-jb/athena-math 2023
4 Multi-view 92.3 ✓ Multi-View Reasoning: Consistent Contrastive Learning for Math Word Problem zwq2018/multi-view-consistency-for-mwp 2022
4 Exp-Tree 92.3 An Expression Tree Decoding Strategy for Mathematical Equation Generation zwq2018/multi-view-consistency-for-mwp 2023
6 ATHENA (roberta-base) 92.2 ATHENA: Mathematical Reasoning with Thought Expansion the-jb/athena-math 2023
7 Roberta-DeductReasoner 92 Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction allanj/deductive-mwp 2022
8 DeBERTa (PM + VM) 91.0 ✓ Math Word Problem Solving by Generating Linguistic Variants of Problem Statements starscream-11813/variational-mathematical-reasoning 2023
9 EPT 88.7 EPT-X: An Expression-Pointer Transformer model that generates eXplanations for numbers snucclab/ept-x
9 Graph2Tree with RoBERTa 88.7 Are NLP Models really able to Solve Simple Math Word Problems? arkilpatel/SVAMP · debjitpaul/refiner · vedantgaur/symbolic-mwp-reasoning 2021
11 GTS with RoBERTa 88.5 Are NLP Models really able to Solve Simple Math Word Problems? arkilpatel/SVAMP · debjitpaul/refiner · vedantgaur/symbolic-mwp-reasoning 2021
12 GEO 85.1 Generating Equation by Utilizing Operators : GEO model 2020
13 EPT-X 84.57 EPT-X: An Expression-Pointer Transformer model that generates eXplanations for numbers snucclab/ept-x
14 EPT 84.51 Point to the Expression: Solving Algebraic Word Problems using the Expression-Pointer Transformer Model snucclab/EPT
15 Graph2Tree 83.7 Graph-to-Tree Learning for Solving Math Word Problems 2003pro/Graph2Tree 2020
16 LLaMA 2-Chat 82.4 Llama 2: Open Foundation and Fine-Tuned Chat Models facebookresearch/llama · llamafamily/llama-chinese · flagalpha/llama2-chinese · +16 2023
17 GPT-3.5 turbo (175B) 80.3 Math Word Problem Solving by Generating Linguistic Variants of Problem Statements starscream-11813/variational-mathematical-reasoning 2023
18 Toolformer 44.0
19 GPT-3 (175B) 19.8
20 Toolformer (disabled) 15.0
1–20 / 100 다음 → 페이지당 10 20 50 100