paper-with-me

Math Word Problem Solving 벤치마크

Math Word Problem Solving on MATH-500

202개 결과 · ⬇ CSV · JSON

Accuracy

5.9 29.27 52.65 76.03 99.4 2022-11 2026-11 GPT-3.5 Turbo (OpenAI) — 44.1 (2022-11-30) GPT-4 (OpenAI) — 56.8 (2023-03-14) Claude Instant (Anthropic) — 26.4 (2023-03-14) Llama 2 Chat 13B (Meta) — 32.9 (2023-07-18) Llama 2 Chat 70B (Meta) — 32.3 (2023-07-18) Llama 2 Chat 7B (Meta) — 5.9 (2023-07-18) Mistral 7B Instruct (Mistral) — 12.1 (2023-09-27) GPT-4 Turbo (OpenAI) — 73.7 (2023-11-06) Claude 2.1 (Anthropic) — 37.4 (2023-11-21) Gemini 1.0 Pro (Google) — 40.3 (2023-12-06) Mistral Medium (Mistral) — 40.5 (2023-12-11) Mixtral 8x7B Instruct (Mistral) — 29.9 (2023-12-11) OpenChat 3.5 (1210) (OpenChat) — 30.7 (2023-12-18) Solar Mini (Upstage) — 33.1 (2024-01-25) Phi-4 Mini Instruct (Microsoft) — 69.6 (2024-02-26) Mistral Small (Feb '24) (Mistral) — 56.2 (2024-02-26) Mistral Large (Feb '24) (Mistral) — 52.7 (2024-02-26) Claude 3 Opus (Anthropic) — 64.1 (2024-03-04) Claude 3 Sonnet (Anthropic) — 41.4 (2024-03-04) Claude 3 Haiku (Anthropic) — 39.4 (2024-03-04) Command-R (Mar '24) (Cohere) — 16.4 (2024-03-12) DBRX Instruct (Databricks) — 27.9 (2024-03-27) Command-R+ (Apr '24) (Cohere) — 27.9 (2024-04-04) Mixtral 8x22B Instruct (Mistral) — 54.5 (2024-04-17) Llama 3 Instruct 8B (Meta) — 49.9 (2024-04-18) Llama 3 Instruct 70B (Meta) — 48.3 (2024-04-18) Phi-3 Mini Instruct 3.8B (Microsoft) — 45.7 (2024-04-23) GPT-4o (May '24) (OpenAI) — 79.1 (2024-05-13) Gemini 1.5 Flash (May '24) (Google) — 55.4 (2024-05-14) Gemini 1.5 Pro (May '24) (Google) — 67.3 (2024-05-15) Qwen2 Instruct 72B (Alibaba) — 70.1 (2024-06-07) DeepSeek-Coder-V2 (DeepSeek) — 74.3 (2024-06-17) Claude 3.5 Sonnet (June '24) (Anthropic) — 69.5 (2024-06-21) GPT-4o mini (OpenAI) — 78.9 (2024-07-18) Llama 3.1 Instruct 405B (Meta) — 70.3 (2024-07-23) Llama 3.1 Instruct 70B (Meta) — 64.9 (2024-07-23) Llama 3.1 Instruct 8B (Meta) — 51.9 (2024-07-23) Mistral Large 2 (Jul '24) (Mistral) — 71.4 (2024-07-24) GPT-4o (Aug '24) (OpenAI) — 79.5 (2024-08-06) Grok Beta (SpaceXAI) — 73.7 (2024-08-13) Hermes 3 - Llama-3.1 70B (Nous Research) — 53.8 (2024-08-15) Jamba 1.5 Large (AI21 Labs) — 60.6 (2024-08-22) Jamba 1.5 Mini (AI21 Labs) — 35.7 (2024-08-22) o1-mini (OpenAI) — 94.4 (2024-09-12) o1-preview (OpenAI) — 92.4 (2024-09-12) Mistral Small (Sep '24) (Mistral) — 56.3 (2024-09-17) Qwen2.5 Instruct 72B (Alibaba) — 85.8 (2024-09-19) Qwen2.5 Instruct 32B (Alibaba) — 80.5 (2024-09-19) Qwen2.5 Coder Instruct 7B (Alibaba) — 66.0 (2024-09-19) Gemini 1.5 Pro (Sep '24) (Google) — 87.6 (2024-09-24) Gemini 1.5 Flash (Sep '24) (Google) — 82.7 (2024-09-24) Llama 3.2 Instruct 90B (Vision) (Meta) — 62.9 (2024-09-25) Llama 3.2 Instruct 11B (Vision) (Meta) — 51.6 (2024-09-25) Llama 3.2 Instruct 3B (Meta) — 48.9 (2024-09-25) Llama 3.2 Instruct 1B (Meta) — 14.0 (2024-09-25) LFM 40B (Liquid AI) — 48.0 (2024-09-30) Gemini 1.5 Flash-8B (Google) — 68.9 (2024-10-03) Reka Flash (Sep '24) (Reka AI) — 52.9 (2024-10-04) Llama 3.1 Nemotron Instruct 70B (NVIDIA) — 73.3 (2024-10-15) Claude 3.5 Sonnet (Oct '24) (Anthropic) — 77.1 (2024-10-22) Claude 3.5 Haiku (Anthropic) — 72.1 (2024-10-22) Qwen2.5 Coder Instruct 32B (Alibaba) — 76.7 (2024-11-11) Qwen2.5 Turbo (Alibaba) — 80.5 (2024-11-18) Mistral Large 2 (Nov '24) (Mistral) — 73.6 (2024-11-18) Pixtral Large (Mistral) — 71.4 (2024-11-18) GPT-4o (Nov '24) (OpenAI) — 75.9 (2024-11-20) QwQ 32B-Preview (Alibaba) — 91.0 (2024-11-27) Nova Pro (Amazon) — 78.6 (2024-12-03) Nova Lite (Amazon) — 76.5 (2024-12-03) Nova Micro (Amazon) — 70.3 (2024-12-03) o1 (OpenAI) — 97.0 (2024-12-05) Llama 3.3 Instruct 70B (Meta) — 77.3 (2024-12-06) DeepSeek-V2.5 (Dec '24) (DeepSeek) — 76.3 (2024-12-10) Gemini 2.0 Flash (experimental) (Google) — 91.1 (2024-12-11) Phi-4 (Microsoft) — 81.0 (2024-12-12) Grok 2 (Dec '24) (SpaceXAI) — 77.8 (2024-12-12) Gemini 2.0 Flash Thinking Experimental (Dec '24) (Google) — 48.0 (2024-12-19) DeepSeek V3 (Dec '24) (DeepSeek) — 88.7 (2024-12-26) DeepSeek R1 (Jan '25) (DeepSeek) — 96.6 (2025-01-20) DeepSeek R1 Distill Qwen 14B (DeepSeek) — 94.9 (2025-01-20) DeepSeek R1 Distill Qwen 32B (DeepSeek) — 94.1 (2025-01-20) DeepSeek R1 Distill Llama 70B (DeepSeek) — 93.5 (2025-01-20) DeepSeek R1 Distill Llama 8B (DeepSeek) — 85.3 (2025-01-20) DeepSeek R1 Distill Qwen 1.5B (DeepSeek) — 68.7 (2025-01-20) Gemini 2.0 Flash Thinking Experimental (Jan '25) (Google) — 94.4 (2025-01-21) Sonar (Perplexity) — 81.7 (2025-01-21) Sonar Pro (Perplexity) — 74.5 (2025-01-21) Sonar Reasoning Pro (Perplexity) — 95.7 (2025-01-28) Sonar Reasoning (Perplexity) — 92.1 (2025-01-28) Qwen2.5 Max (Alibaba) — 83.5 (2025-01-28) Llama 3.1 Tulu3 405B (Allen Institute for AI) — 77.8 (2025-01-30) Mistral Small 3 (Mistral) — 71.5 (2025-01-30) o3-mini (High) (OpenAI) — 98.5 (2025-01-31) o3-mini (OpenAI) — 97.3 (2025-01-31) Gemini 2.0 Flash (Feb '25) (Google) — 93.0 (2025-02-05) Gemini 2.0 Pro Experimental (Feb '25) (Google) — 92.3 (2025-02-05) Gemini 2.0 Flash-Lite (Preview) (Google) — 87.3 (2025-02-05) DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) (Nous Research) — 21.8 (2025-02-13) GPT-4o (ChatGPT) (OpenAI) — 79.7 (2025-02-15) Mistral Saba (Mistral) — 67.7 (2025-02-17) R1 1776 (Perplexity) — 95.4 (2025-02-18) Grok 3 mini Reasoning (High) (SpaceXAI) — 99.2 (2025-02-19) Grok 3 (SpaceXAI) — 87.0 (2025-02-19) Claude 3.7 Sonnet (Reasoning) (Anthropic) — 94.7 (2025-02-24) Claude 3.7 Sonnet (Non-reasoning) (Anthropic) — 85.0 (2025-02-24) Gemini 2.0 Flash-Lite (Feb '25) (Google) — 87.3 (2025-02-25) Phi-4 Multimodal Instruct (Microsoft) — 69.3 (2025-02-26) QwQ 32B (Alibaba) — 95.7 (2025-03-05) Jamba 1.6 Large (AI21 Labs) — 58.0 (2025-03-06) Jamba 1.6 Mini (AI21 Labs) — 25.7 (2025-03-06) Reka Flash 3 (Reka AI) — 89.3 (2025-03-10) Gemma 3 27B Instruct (Google) — 88.3 (2025-03-12) Gemma 3 12B Instruct (Google) — 85.3 (2025-03-12) Gemma 3 4B Instruct (Google) — 76.6 (2025-03-12) Command A (Cohere) — 81.9 (2025-03-13) DeepHermes 3 - Mistral 24B Preview (Non-reasoning) (Nous Research) — 59.5 (2025-03-13) Gemma 3 1B Instruct (Google) — 48.4 (2025-03-13) Mistral Small 3.1 (Mistral) — 70.7 (2025-03-17) Llama 3.3 Nemotron Super 49B v1 (Reasoning) (NVIDIA) — 95.9 (2025-03-18) Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) (NVIDIA) — 77.5 (2025-03-18) Gemini 2.5 Pro Preview (Mar' 25) (Google) — 98.0 (2025-03-25) DeepSeek V3 0324 (DeepSeek) — 94.2 (2025-03-25) GPT-4o (March 2025, chatgpt-4o-latest) (OpenAI) — 89.3 (2025-03-27) Llama 4 Maverick (Meta) — 88.9 (2025-04-05) Llama 4 Scout (Meta) — 84.4 (2025-04-05) Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) (NVIDIA) — 95.2 (2025-04-07) GPT-4.1 mini (OpenAI) — 92.5 (2025-04-14) GPT-4.1 (OpenAI) — 91.3 (2025-04-14) GPT-4.1 nano (OpenAI) — 84.8 (2025-04-14) o3 (OpenAI) — 99.2 (2025-04-16) o4-mini (High) (OpenAI) — 98.9 (2025-04-16) Granite 3.3 8B (Non-reasoning) (IBM) — 66.5 (2025-04-16) Gemini 2.5 Flash Preview (Reasoning) (Google) — 98.1 (2025-04-17) Gemini 2.5 Flash Preview (Non-reasoning) (Google) — 92.6 (2025-04-17) Qwen3 32B (Reasoning) (Alibaba) — 96.1 (2025-04-28) Qwen3 14B (Reasoning) (Alibaba) — 96.1 (2025-04-28) Qwen3 30B A3B (Reasoning) (Alibaba) — 95.9 (2025-04-28) Qwen3 4B (Reasoning) (Alibaba) — 93.3 (2025-04-28) Qwen3 235B A22B (Reasoning) (Alibaba) — 93.0 (2025-04-28) Qwen3 8B (Reasoning) (Alibaba) — 90.4 (2025-04-28) Qwen3 235B A22B (Non-reasoning) (Alibaba) — 90.2 (2025-04-28) Qwen3 1.7B (Reasoning) (Alibaba) — 89.4 (2025-04-28) Qwen3 14B (Non-reasoning) (Alibaba) — 87.1 (2025-04-28) Qwen3 32B (Non-reasoning) (Alibaba) — 86.9 (2025-04-28) Qwen3 30B A3B (Non-reasoning) (Alibaba) — 86.3 (2025-04-28) Qwen3 4B (Non-reasoning) (Alibaba) — 84.3 (2025-04-28) Qwen3 8B (Non-reasoning) (Alibaba) — 82.8 (2025-04-28) Qwen3 0.6B (Reasoning) (Alibaba) — 75.0 (2025-04-28) Qwen3 1.7B (Non-reasoning) (Alibaba) — 71.7 (2025-04-28) Qwen3 0.6B (Non-reasoning) (Alibaba) — 52.1 (2025-04-28) Nova Premier (Amazon) — 83.9 (2025-04-30) Gemini 2.5 Pro Preview (May' 25) (Google) — 98.6 (2025-05-06) Mistral Medium 3 (Mistral) — 90.7 (2025-05-07) Gemini 2.5 Flash (Reasoning) (Google) — 98.1 (2025-05-20) Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) (NVIDIA) — 94.7 (2025-05-20) Gemini 2.5 Flash (Non-reasoning) (Google) — 93.2 (2025-05-20) Solar Pro 2 (Preview) (Reasoning) (Upstage) — 90.0 (2025-05-20) Solar Pro 2 (Preview) (Non-reasoning) (Upstage) — 87.1 (2025-05-20) Gemma 3n E4B Instruct Preview (May '25) (Google) — 74.9 (2025-05-20) Devstral Small (May '25) (Mistral) — 68.4 (2025-05-21) Claude 4 Sonnet (Reasoning) (Anthropic) — 99.1 (2025-05-22) Claude 4 Opus (Reasoning) (Anthropic) — 98.2 (2025-05-22) Claude 4 Opus (Non-reasoning) (Anthropic) — 94.1 (2025-05-22) Claude 4 Sonnet (Non-reasoning) (Anthropic) — 93.4 (2025-05-22) Sarvam M (Reasoning, Based on Mistral Small 3.1) (Sarvam) — 84.7 (2025-05-23) DeepSeek R1 0528 (May '25) (DeepSeek) — 98.3 (2025-05-28) DeepSeek R1 0528 Qwen3 8B (DeepSeek) — 93.2 (2025-05-29) Gemini 2.5 Pro (Google) — 96.7 (2025-06-05) Magistral Small 1 (Mistral) — 96.3 (2025-06-10) Magistral Medium 1 (Mistral) — 91.7 (2025-06-10) MiniMax M1 80k (MiniMax) — 98.0 (2025-06-17) MiniMax M1 40k (MiniMax) — 97.2 (2025-06-17) Gemini 2.5 Flash-Lite (Reasoning) (Google) — 96.9 (2025-06-17) Gemini 2.5 Flash-Lite (Non-reasoning) (Google) — 92.6 (2025-06-17) Mistral Small 3.2 (Mistral) — 88.3 (2025-06-20) Gemma 3n E4B Instruct (Google) — 77.1 (2025-06-26) Gemma 3n E2B Instruct (Google) — 69.1 (2025-06-26) ERNIE 4.5 300B A47B (Baidu) — 93.1 (2025-06-30) Jamba 1.7 Large (AI21 Labs) — 60.0 (2025-07-07) Jamba 1.7 Mini (AI21 Labs) — 25.8 (2025-07-07) Solar Pro 2 (Reasoning) (Upstage) — 96.7 (2025-07-09) Solar Pro 2 (Non-reasoning) (Upstage) — 88.9 (2025-07-09) Grok 4 (SpaceXAI) — 99.0 (2025-07-10) Devstral Medium (Mistral) — 70.7 (2025-07-10) Devstral Small (Jul '25) (Mistral) — 63.5 (2025-07-10) Kimi K2 (Kimi) — 97.1 (2025-07-11) EXAONE 4.0 32B (Reasoning) (LG AI Research) — 97.7 (2025-07-15) EXAONE 4.0 32B (Non-reasoning) (LG AI Research) — 93.9 (2025-07-15) Qwen3 235B A22B 2507 Instruct (Alibaba) — 98.0 (2025-07-21) Qwen3 Coder 480B A35B Instruct (Alibaba) — 94.2 (2025-07-22) Qwen3 235B A22B 2507 (Reasoning) (Alibaba) — 98.4 (2025-07-25) Llama Nemotron Super 49B v1.5 (Reasoning) (NVIDIA) — 98.3 (2025-07-25) Llama Nemotron Super 49B v1.5 (Non-reasoning) (NVIDIA) — 77.0 (2025-07-25) GLM-4.5 (Reasoning) (Z AI) — 97.9 (2025-07-28) GLM-4.5-Air (Z AI) — 96.5 (2025-07-28) Qwen3 30B A3B 2507 Instruct (Alibaba) — 97.5 (2025-07-29) Qwen3 30B A3B 2507 (Reasoning) (Alibaba) — 97.6 (2025-07-30) Qwen3 Coder 30B A3B Instruct (Alibaba) — 89.3 (2025-07-31) GPT-5 (High) (OpenAI) — 99.4 (2025-08-07) GPT-5 (Medium) (OpenAI) — 99.1 (2025-08-07) GPT-5 (Low) (OpenAI) — 98.7 (2025-08-07) GPT-5 (Minimal) (OpenAI) — 86.1 (2025-08-07) GPT-3.5 Turbo (OpenAI) — 44.1 (2022-11-30) GPT-4 (OpenAI) — 56.8 (2023-03-14) GPT-4 Turbo (OpenAI) — 73.7 (2023-11-06) GPT-4o (May '24) (OpenAI) — 79.1 (2024-05-13) GPT-4o (Aug '24) (OpenAI) — 79.5 (2024-08-06) o1-mini (OpenAI) — 94.4 (2024-09-12) o1 (OpenAI) — 97.0 (2024-12-05) o3-mini (High) (OpenAI) — 98.5 (2025-01-31) Grok 3 mini Reasoning (High) (SpaceXAI) — 99.2 (2025-02-19) GPT-5 (High) (OpenAI) — 99.4 (2025-08-07)
RankModel Accuracy PaperCodeYear
1 GPT-5 (High) (OpenAI) 외부 99.4 artificialanalysis.ai 2025
2 o3 (OpenAI) 외부 99.2 artificialanalysis.ai 2025
2 Grok 3 mini Reasoning (High) (SpaceXAI) 외부 99.2 artificialanalysis.ai 2025
4 GPT-5 (Medium) (OpenAI) 외부 99.1 artificialanalysis.ai 2025
4 Claude 4 Sonnet (Reasoning) (Anthropic) 외부 99.1 artificialanalysis.ai 2025
6 Grok 4 (SpaceXAI) 외부 99 artificialanalysis.ai 2025
7 o4-mini (High) (OpenAI) 외부 98.9 artificialanalysis.ai 2025
8 GPT-5 (Low) (OpenAI) 외부 98.7 artificialanalysis.ai 2025
9 Gemini 2.5 Pro Preview (May' 25) (Google) 외부 98.6 artificialanalysis.ai 2025
10 o3-mini (High) (OpenAI) 외부 98.5 artificialanalysis.ai 2025
11 Qwen3 235B A22B 2507 (Reasoning) (Alibaba) 외부 98.4 artificialanalysis.ai 2025
12 DeepSeek R1 0528 (May '25) (DeepSeek) 외부 98.3 artificialanalysis.ai 2025
12 Llama Nemotron Super 49B v1.5 (Reasoning) (NVIDIA) 외부 98.3 artificialanalysis.ai 2025
14 Claude 4 Opus (Reasoning) (Anthropic) 외부 98.2 artificialanalysis.ai 2025
15 Gemini 2.5 Flash (Reasoning) (Google) 외부 98.1 artificialanalysis.ai 2025
15 Gemini 2.5 Flash Preview (Reasoning) (Google) 외부 98.1 artificialanalysis.ai 2025
17 Gemini 2.5 Pro Preview (Mar' 25) (Google) 외부 98 artificialanalysis.ai 2025
17 Qwen3 235B A22B 2507 Instruct (Alibaba) 외부 98 artificialanalysis.ai 2025
17 MiniMax M1 80k (MiniMax) 외부 98 artificialanalysis.ai 2025
20 GLM-4.5 (Reasoning) (Z AI) 외부 97.9 artificialanalysis.ai 2025
1–20 / 202 다음 → 페이지당 10 20 50 100