paper-with-me

Code Generation 벤치마크

Code Generation on Terminal-Bench Hard

432개 결과 · ⬇ CSV · JSON

Accuracy

0 16.48 32.95 49.43 65.9 2024-02 2026-09 Phi-4 Mini Instruct (Microsoft) — 0.0 (2024-02-26) Claude 3 Haiku (Anthropic) — 0.8 (2024-03-04) Llama 3 Instruct 70B (Meta) — 0.8 (2024-04-18) Llama 3 Instruct 8B (Meta) — 0.0 (2024-04-18) Phi-3 Mini Instruct 3.8B (Microsoft) — 0.0 (2024-04-23) Llama 3.1 Instruct 405B (Meta) — 6.8 (2024-07-23) Llama 3.1 Instruct 70B (Meta) — 3.0 (2024-07-23) Llama 3.1 Instruct 8B (Meta) — 0.8 (2024-07-23) GPT-4o (Aug '24) (OpenAI) — 8.3 (2024-08-06) Qwen2.5 Instruct 72B (Alibaba) — 4.5 (2024-09-19) Llama 3.2 Instruct 11B (Vision) (Meta) — 0.8 (2024-09-25) Molmo 7B-D (Allen Institute for AI) — 0.0 (2024-09-25) Llama 3.2 Instruct 1B (Meta) — 0.0 (2024-09-25) Llama 3.1 Nemotron Instruct 70B (NVIDIA) — 4.5 (2024-10-15) Claude 3.5 Haiku (Anthropic) — 2.3 (2024-10-22) Mistral Large 2 (Nov '24) (Mistral) — 6.1 (2024-11-18) GPT-4o (Nov '24) (OpenAI) — 8.3 (2024-11-20) OLMo 2 7B (Allen Institute for AI) — 0.0 (2024-11-26) Nova Pro (Amazon) — 6.1 (2024-12-03) Nova Micro (Amazon) — 1.5 (2024-12-03) Nova Lite (Amazon) — 0.8 (2024-12-03) o1 (OpenAI) — 12.9 (2024-12-05) Llama 3.3 Instruct 70B (Meta) — 3.0 (2024-12-06) Phi-4 (Microsoft) — 3.8 (2024-12-12) DeepSeek V3 (Dec '24) (DeepSeek) — 6.8 (2024-12-26) DeepSeek R1 (Jan '25) (DeepSeek) — 6.1 (2025-01-20) DeepSeek R1 Distill Llama 70B (DeepSeek) — 1.5 (2025-01-20) o3-mini (OpenAI) — 6.8 (2025-01-31) o3-mini (high) (OpenAI) — 6.1 (2025-01-31) Gemini 2.0 Flash (Feb '25) (Google) — 3.8 (2025-02-05) Grok 3 mini Reasoning (high) (SpaceXAI) — 17.4 (2025-02-19) Grok 3 (SpaceXAI) — 11.4 (2025-02-19) Claude 3.7 Sonnet (Reasoning) (Anthropic) — 21.2 (2025-02-24) Claude 3.7 Sonnet (Non-reasoning) (Anthropic) — 21.2 (2025-02-24) Reka Flash 3 (Reka AI) — 0.0 (2025-03-10) Gemma 3 27B Instruct (Google) — 3.8 (2025-03-12) Gemma 3 4B Instruct (Google) — 0.8 (2025-03-12) Gemma 3 12B Instruct (Google) — 0.8 (2025-03-12) Command A (Cohere) — 0.8 (2025-03-13) OLMo 2 32B (Allen Institute for AI) — 0.0 (2025-03-13) Gemma 3 1B Instruct (Google) — 0.0 (2025-03-13) Mistral Small 3.1 (Mistral) — 7.6 (2025-03-17) Llama 3.3 Nemotron Super 49B v1 (Reasoning) (NVIDIA) — 0.0 (2025-03-18) Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) (NVIDIA) — 0.0 (2025-03-18) DeepSeek V3 0324 (DeepSeek) — 15.2 (2025-03-25) Llama 4 Maverick (Meta) — 6.8 (2025-04-05) Llama 4 Scout (Meta) — 1.5 (2025-04-05) Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) (NVIDIA) — 2.3 (2025-04-07) GPT-4.1 (OpenAI) — 13.6 (2025-04-14) GPT-4.1 mini (OpenAI) — 7.6 (2025-04-14) GPT-4.1 nano (OpenAI) — 3.8 (2025-04-14) o3 (OpenAI) — 37.1 (2025-04-16) o4-mini (high) (OpenAI) — 15.2 (2025-04-16) Granite 3.3 8B (Non-reasoning) (IBM) — 0.0 (2025-04-16) Qwen3 30B A3B (Non-reasoning) (Alibaba) — 6.8 (2025-04-28) Qwen3 235B A22B (Reasoning) (Alibaba) — 6.1 (2025-04-28) Qwen3 235B A22B (Non-reasoning) (Alibaba) — 6.1 (2025-04-28) Qwen3 14B (Non-reasoning) (Alibaba) — 5.3 (2025-04-28) Qwen3 14B (Reasoning) (Alibaba) — 3.8 (2025-04-28) Qwen3 32B (Reasoning) (Alibaba) — 3.0 (2025-04-28) Qwen3 30B A3B (Reasoning) (Alibaba) — 2.3 (2025-04-28) Qwen3 8B (Non-reasoning) (Alibaba) — 2.3 (2025-04-28) Qwen3 8B (Reasoning) (Alibaba) — 2.3 (2025-04-28) Qwen3 1.7B (Reasoning) (Alibaba) — 0.0 (2025-04-28) Qwen3 1.7B (Non-reasoning) (Alibaba) — 0.0 (2025-04-28) Qwen3 0.6B (Non-reasoning) (Alibaba) — 0.0 (2025-04-28) Qwen3 0.6B (Reasoning) (Alibaba) — 0.0 (2025-04-28) Nova Premier (Amazon) — 6.8 (2025-04-30) Mistral Medium 3 (Mistral) — 3.8 (2025-05-07) Gemini 2.5 Flash (Reasoning) (Google) — 13.6 (2025-05-20) Gemini 2.5 Flash (Non-reasoning) (Google) — 12.1 (2025-05-20) Devstral Small (May '25) (Mistral) — 6.1 (2025-05-21) Claude 4 Opus (Reasoning) (Anthropic) — 31.1 (2025-05-22) Claude 4 Sonnet (Reasoning) (Anthropic) — 31.1 (2025-05-22) Claude 4 Sonnet (Non-reasoning) (Anthropic) — 27.3 (2025-05-22) Sarvam M (Reasoning) (Sarvam) — 2.3 (2025-05-23) DeepSeek R1 0528 (May '25) (DeepSeek) — 15.9 (2025-05-28) DeepSeek R1 0528 Qwen3 8B (DeepSeek) — 1.5 (2025-05-29) Gemini 2.5 Pro (Google) — 26.5 (2025-06-05) Magistral Medium 1 (Mistral) — 9.1 (2025-06-10) Magistral Small 1 (Mistral) — 4.5 (2025-06-10) Gemini 2.5 Flash-Lite (Reasoning) (Google) — 4.5 (2025-06-17) MiniMax M1 80k (MiniMax) — 3.0 (2025-06-17) MiniMax M1 40k (MiniMax) — 2.3 (2025-06-17) Gemini 2.5 Flash-Lite (Non-reasoning) (Google) — 2.3 (2025-06-17) Mistral Small 3.2 (Mistral) — 6.8 (2025-06-20) Gemma 3n E4B Instruct (Google) — 2.3 (2025-06-26) Gemma 3n E2B Instruct (Google) — 0.8 (2025-06-26) ERNIE 4.5 300B A47B (Baidu) — 6.1 (2025-06-30) Jamba 1.7 Large (AI21 Labs) — 2.3 (2025-07-07) Jamba 1.7 Mini (AI21 Labs) — 0.0 (2025-07-07) Solar Pro 2 (Non-reasoning) (Upstage) — 4.5 (2025-07-09) Solar Pro 2 (Reasoning) (Upstage) — 3.0 (2025-07-09) Grok 4 (SpaceXAI) — 37.9 (2025-07-10) Devstral Medium (Mistral) — 9.1 (2025-07-10) Devstral Small (Jul '25) (Mistral) — 6.1 (2025-07-10) LFM2 1.2B (Liquid AI) — 0.0 (2025-07-10) Kimi K2 (Kimi) — 15.9 (2025-07-11) EXAONE 4.0 32B (Reasoning) (LG AI Research) — 3.8 (2025-07-15) EXAONE 4.0 32B (Non-reasoning) (LG AI Research) — 1.5 (2025-07-15) Exaone 4.0 1.2B (Reasoning) (LG AI Research) — 0.0 (2025-07-15) Exaone 4.0 1.2B (Non-reasoning) (LG AI Research) — 0.0 (2025-07-15) Qwen3 235B A22B 2507 Instruct (Alibaba) — 15.2 (2025-07-21) Qwen3 Coder 480B A35B Instruct (Alibaba) — 18.9 (2025-07-22) Qwen3 235B A22B 2507 (Reasoning) (Alibaba) — 13.6 (2025-07-25) Llama Nemotron Super 49B v1.5 (Reasoning) (NVIDIA) — 5.3 (2025-07-25) Llama Nemotron Super 49B v1.5 (Non-reasoning) (NVIDIA) — 3.8 (2025-07-25) GLM-4.5 (Reasoning) (Z AI) — 22.0 (2025-07-28) GLM-4.5-Air (Z AI) — 20.5 (2025-07-28) Qwen3 30B A3B 2507 Instruct (Alibaba) — 6.1 (2025-07-29) Qwen3 30B A3B 2507 (Reasoning) (Alibaba) — 5.3 (2025-07-30) Qwen3 Coder 30B A3B Instruct (Alibaba) — 15.2 (2025-07-31) Claude 4.1 Opus (Reasoning) (Anthropic) — 34.3 (2025-08-05) gpt-oss-120b (high) (OpenAI) — 23.5 (2025-08-05) gpt-oss-20b (high) (OpenAI) — 10.6 (2025-08-05) gpt-oss-120b (low) (OpenAI) — 5.3 (2025-08-05) gpt-oss-20b (low) (OpenAI) — 4.5 (2025-08-05) Qwen3 4B 2507 Instruct (Alibaba) — 4.5 (2025-08-06) Qwen3 4B 2507 (Reasoning) (Alibaba) — 1.5 (2025-08-06) GPT-5 (medium) (OpenAI) — 37.9 (2025-08-07) GPT-5 mini (high) (OpenAI) — 33.3 (2025-08-07) GPT-5 (high) (OpenAI) — 32.6 (2025-08-07) GPT-5 mini (medium) (OpenAI) — 28.8 (2025-08-07) GPT-5 (low) (OpenAI) — 26.5 (2025-08-07) GPT-5 (minimal) (OpenAI) — 18.2 (2025-08-07) GPT-5 nano (medium) (OpenAI) — 17.4 (2025-08-07) GPT-5 mini (minimal) (OpenAI) — 14.4 (2025-08-07) GPT-5 (ChatGPT) (OpenAI) — 12.9 (2025-08-07) GPT-5 nano (high) (OpenAI) — 12.1 (2025-08-07) GPT-5 nano (minimal) (OpenAI) — 6.8 (2025-08-07) GLM-4.5V (Non-reasoning) (Z AI) — 6.8 (2025-08-11) GLM-4.5V (Reasoning) (Z AI) — 5.3 (2025-08-11) Mistral Medium 3.1 (Mistral) — 10.6 (2025-08-12) Gemma 3 270M (Google) — 0.0 (2025-08-14) NVIDIA Nemotron Nano 9B V2 (Reasoning) (NVIDIA) — 1.5 (2025-08-18) NVIDIA Nemotron Nano 9B V2 (Non-reasoning) (NVIDIA) — 0.8 (2025-08-18) Seed-OSS-36B-Instruct (ByteDance Seed) — 6.8 (2025-08-20) DeepSeek V3.1 (Reasoning) (DeepSeek) — 25.0 (2025-08-21) DeepSeek V3.1 (Non-reasoning) (DeepSeek) — 24.2 (2025-08-21) Hermes 4 - Llama-3.1 405B (Reasoning) (Nous Research) — 11.4 (2025-08-27) Hermes 4 - Llama-3.1 405B (Non-reasoning) (Nous Research) — 9.8 (2025-08-27) Hermes 4 - Llama-3.1 70B (Reasoning) (Nous Research) — 4.5 (2025-08-27) Hermes 4 - Llama-3.1 70B (Non-reasoning) (Nous Research) — 0.0 (2025-08-27) Grok Code Fast 1 (SpaceXAI) — 17.4 (2025-08-28) Apertus 70B Instruct (Swiss AI Initiative) — 0.0 (2025-09-02) Apertus 8B Instruct (Swiss AI Initiative) — 0.0 (2025-09-02) Kimi K2 0905 (Kimi) — 23.5 (2025-09-05) Qwen3 Max (Preview) (Alibaba) — 19.7 (2025-09-05) Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning) (Google) — 12.9 (2025-09-08) Ling-mini-2.0 (InclusionAI) — 0.8 (2025-09-09) Qwen3 Next 80B A3B (Reasoning) (Alibaba) — 9.8 (2025-09-11) Qwen3 Next 80B A3B Instruct (Alibaba) — 7.6 (2025-09-11) Ling-flash-2.0 (InclusionAI) — 10.6 (2025-09-17) Magistral Small 1.2 (Mistral) — 4.5 (2025-09-17) Magistral Medium 1.2 (Mistral) — 12.9 (2025-09-18) Grok 4 Fast (Reasoning) (SpaceXAI) — 18.9 (2025-09-19) Grok 4 Fast (Non-reasoning) (SpaceXAI) — 12.1 (2025-09-19) Ring-flash-2.0 (InclusionAI) — 7.6 (2025-09-19) DeepSeek V3.1 Terminus (Non-reasoning) (DeepSeek) — 31.8 (2025-09-22) DeepSeek V3.1 Terminus (Reasoning) (DeepSeek) — 30.3 (2025-09-22) Qwen3 Omni 30B A3B (Reasoning) (Alibaba) — 3.8 (2025-09-22) Granite 4.0 H Small (IBM) — 2.3 (2025-09-22) Qwen3 Omni 30B A3B Instruct (Alibaba) — 1.5 (2025-09-22) Granite 4.0 Micro (IBM) — 1.5 (2025-09-22) GPT-5 Codex (high) (OpenAI) — 37.9 (2025-09-23) Qwen3 Max (Alibaba) — 20.5 (2025-09-23) Qwen3 VL 235B A22B (Reasoning) (Alibaba) — 11.4 (2025-09-23) Qwen3 VL 235B A22B Instruct (Alibaba) — 6.8 (2025-09-23) LFM2 2.6B (Liquid AI) — 0.8 (2025-09-23) Gemini 2.5 Flash Preview (Sep '25) (Reasoning) (Google) — 16.7 (2025-09-25) Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning) (Google) — 14.4 (2025-09-25) Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning) (Google) — 7.6 (2025-09-25) Claude 4.5 Sonnet (Reasoning) (Anthropic) — 35.6 (2025-09-29) DeepSeek V3.2 Exp (Reasoning) (DeepSeek) — 31.1 (2025-09-29) Claude 4.5 Sonnet (Non-reasoning) (Anthropic) — 28.8 (2025-09-29) DeepSeek V3.2 Exp (Non-reasoning) (DeepSeek) — 25.0 (2025-09-29) GLM-4.6 (Non-reasoning) (Z AI) — 28.8 (2025-09-30) GLM-4.6 (Reasoning) (Z AI) — 25.0 (2025-09-30) Apriel-v1.5-15B-Thinker (ServiceNow) — 10.6 (2025-09-30) Qwen3 VL 30B A3B Instruct (Alibaba) — 6.1 (2025-10-03) Qwen3 VL 30B A3B (Reasoning) (Alibaba) — 5.3 (2025-10-03) LFM2 8B A1B (Liquid AI) — 0.0 (2025-10-07) Ling-1T (InclusionAI) — 10.6 (2025-10-08) Jamba Reasoning 3B (AI21 Labs) — 0.8 (2025-10-08) Ring-1T (InclusionAI) — 6.8 (2025-10-13) Qwen3 VL 8B (Reasoning) (Alibaba) — 3.8 (2025-10-14) Qwen3 VL 8B Instruct (Alibaba) — 2.3 (2025-10-14) Qwen3 VL 4B (Reasoning) (Alibaba) — 1.5 (2025-10-14) Qwen3 VL 4B Instruct (Alibaba) — 0.0 (2025-10-14) Claude 4.5 Haiku (Reasoning) (Anthropic) — 27.3 (2025-10-15) Claude 4.5 Haiku (Non-reasoning) (Anthropic) — 27.3 (2025-10-15) Qwen3 VL 32B Instruct (Alibaba) — 8.3 (2025-10-21) Qwen3 VL 32B (Reasoning) (Alibaba) — 7.6 (2025-10-21) MiniMax-M2 (MiniMax) — 25.8 (2025-10-26) NVIDIA Nemotron Nano 12B v2 VL (Reasoning) (NVIDIA) — 4.5 (2025-10-28) NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning) (NVIDIA) — 0.0 (2025-10-28) Granite 4.0 H 1B (IBM) — 0.0 (2025-10-28) Granite 4.0 1B (IBM) — 0.0 (2025-10-28) Granite 4.0 350M (IBM) — 0.0 (2025-10-28) Granite 4.0 H 350M (IBM) — 0.0 (2025-10-28) Nova 2.0 Lite (medium) (Amazon) — 17.4 (2025-10-29) Nova 2.0 Lite (high) (Amazon) — 16.7 (2025-10-29) Nova 2.0 Lite (Non-reasoning) (Amazon) — 6.8 (2025-10-29) Nova 2.0 Lite (low) (Amazon) — 3.8 (2025-10-29) Kimi Linear 48B A3B Instruct (Kimi) — 11.4 (2025-10-30) Qwen3 Max Thinking (Preview) (Alibaba) — 17.4 (2025-11-03) Kimi K2 Thinking (Kimi) — 31.1 (2025-11-06) Doubao Seed Code (ByteDance Seed) — 26.5 (2025-11-11) KAT-Coder-Pro V1 (KwaiKAT) — 9.1 (2025-11-11) GPT-5.1 (high) (OpenAI) — 45.5 (2025-11-13) GPT-5.1 Codex (high) (OpenAI) — 34.8 (2025-11-13) GPT-5.1 Codex mini (high) (OpenAI) — 33.3 (2025-11-13) ERNIE 5.0 Thinking Preview (Baidu) — 25.0 (2025-11-13) GPT-5.1 (Non-reasoning) (OpenAI) — 22.7 (2025-11-13) Gemini 3 Pro Preview (high) (Google) — 41.7 (2025-11-18) Gemini 3 Pro Preview (low) (Google) — 34.1 (2025-11-18) Cogito v2.1 (Reasoning) (Deep Cogito) — 16.7 (2025-11-18) Grok 4.1 Fast (Reasoning) (SpaceXAI) — 24.2 (2025-11-19) Grok 4.1 Fast (Non-reasoning) (SpaceXAI) — 14.4 (2025-11-19) Olmo 3 32B Think (Allen Institute for AI) — 1.5 (2025-11-20) Olmo 3 7B Think (Allen Institute for AI) — 0.8 (2025-11-20) Olmo 3 7B Instruct (Allen Institute for AI) — 0.0 (2025-11-20) Claude Opus 4.5 (Reasoning) (Anthropic) — 47.0 (2025-11-24) Claude Opus 4.5 (Non-reasoning) (Anthropic) — 40.9 (2025-11-24) Apriel-v1.6-15B-Thinker (ServiceNow) — 14.4 (2025-11-25) Nova 2.0 Omni (Non-reasoning) (Amazon) — 6.8 (2025-11-26) Nova 2.0 Omni (medium) (Amazon) — 4.5 (2025-11-26) Nova 2.0 Omni (low) (Amazon) — 3.8 (2025-11-26) Nova 2.0 Pro Preview (medium) (Amazon) — 24.2 (2025-11-27) Nova 2.0 Pro Preview (low) (Amazon) — 17.4 (2025-11-27) Nova 2.0 Pro Preview (Non-reasoning) (Amazon) — 16.7 (2025-11-27) INTELLECT-3 (Prime Intellect) — 9.1 (2025-11-27) DeepSeek V3.2 (Reasoning) (DeepSeek) — 35.6 (2025-12-01) DeepSeek V3.2 Speciale (DeepSeek) — 34.8 (2025-12-01) DeepSeek V3.2 (Non-reasoning) (DeepSeek) — 32.6 (2025-12-01) Mistral Large 3 (Mistral) — 15.9 (2025-12-02) Ministral 3 14B (Mistral) — 4.5 (2025-12-02) Ministral 3 8B (Mistral) — 4.5 (2025-12-02) Ministral 3 3B (Mistral) — 0.0 (2025-12-02) Motif-2-12.7B-Reasoning (Motif Technologies) — 3.8 (2025-12-04) K2-V2 (high) (MBZUAI Institute of Foundation Models) — 9.8 (2025-12-05) K2-V2 (medium) (MBZUAI Institute of Foundation Models) — 8.3 (2025-12-05) K2-V2 (low) (MBZUAI Institute of Foundation Models) — 4.5 (2025-12-05) GLM-4.6V (Reasoning) (Z AI) — 14.4 (2025-12-08) GLM-4.6V (Non-reasoning) (Z AI) — 3.0 (2025-12-08) Devstral 2 (Mistral) — 18.9 (2025-12-09) Devstral Small 2 (Mistral) — 16.7 (2025-12-09) GPT-5.2 (xhigh) (OpenAI) — 47.0 (2025-12-11) GPT-5.2 (medium) (OpenAI) — 43.2 (2025-12-11) GPT-5.2 Codex (xhigh) (OpenAI) — 37.1 (2025-12-11) GPT-5.2 (Non-reasoning) (OpenAI) — 31.8 (2025-12-11) Mi:dm K 2.5 Pro Preview (Korea Telecom) — 3.0 (2025-12-11) Mi:dm K 2.5 Pro (Korea Telecom) — 2.3 (2025-12-11) Molmo2-8B (Allen Institute for AI) — 0.0 (2025-12-11) Olmo 3.1 32B Think (Allen Institute for AI) — 0.0 (2025-12-12) NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) (NVIDIA) — 13.6 (2025-12-15) NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) (NVIDIA) — 12.1 (2025-12-15) K2 Think V2 (MBZUAI Institute of Foundation Models) — 6.8 (2025-12-15) MiMo-V2-Flash (Feb 2026) (Xiaomi) — 31.1 (2025-12-16) MiMo-V2-Flash (Reasoning) (Xiaomi) — 28.0 (2025-12-16) MiMo-V2-Flash (Non-reasoning) (Xiaomi) — 25.8 (2025-12-16) Gemini 3 Flash Preview (Reasoning) (Google) — 38.6 (2025-12-17) Gemini 3 Flash Preview (Non-reasoning) (Google) — 31.8 (2025-12-17) Solar Open 100B (Reasoning) (Upstage) — 2.3 (2025-12-17) GLM-4.7 (Reasoning) (Z AI) — 31.8 (2025-12-22) GLM-4.7 (Non-reasoning) (Z AI) — 30.3 (2025-12-22) MiniMax-M2.1 (MiniMax) — 28.8 (2025-12-23) HyperCLOVA X SEED Think (32B) (Naver) — 12.1 (2025-12-26) K-EXAONE (Reasoning) (LG AI Research) — 22.7 (2025-12-31) K-EXAONE (Non-reasoning) (LG AI Research) — 6.8 (2025-12-31) Falcon-H1R-7B (TII UAE) — 2.3 (2026-01-04) LFM2.5-1.2B-Instruct (Liquid AI) — 0.0 (2026-01-05) LFM2.5-VL-1.6B (Liquid AI) — 0.0 (2026-01-05) Olmo 3.1 32B Instruct (Allen Institute for AI) — 0.0 (2026-01-13) GLM-4.7-Flash (Reasoning) (Z AI) — 22.0 (2026-01-19) GLM-4.7-Flash (Non-reasoning) (Z AI) — 3.8 (2026-01-19) Step3 VL 10B (StepFun) — 5.3 (2026-01-20) LFM2.5-1.2B-Thinking (Liquid AI) — 0.0 (2026-01-20) Qwen3 Max Thinking (Alibaba) — 24.2 (2026-01-26) Kimi K2.5 (Reasoning) (Kimi) — 34.8 (2026-01-27) Kimi K2.5 (Non-reasoning) (Kimi) — 18.9 (2026-01-27) LongCat Flash Lite (LongCat) — 10.6 (2026-01-28) Step 3.5 Flash (StepFun) — 27.3 (2026-02-02) Qwen3 Coder Next (Alibaba) — 18.2 (2026-02-03) GPT-5.3 Codex (xhigh) (OpenAI) — 53.0 (2026-02-05) Claude Opus 4.6 (Non-reasoning, High Effort) (Anthropic) — 48.5 (2026-02-05) Claude Opus 4.6 (Adaptive Reasoning, Max Effort) (Anthropic) — 46.2 (2026-02-05) Tri-21B-think Preview (Trillion Labs) — 2.3 (2026-02-10) Tri-21B-Think (Trillion Labs) — 0.8 (2026-02-10) GLM-5 (Reasoning) (Z AI) — 43.2 (2026-02-11) GLM-5 (Non-reasoning) (Z AI) — 39.4 (2026-02-11) Nanbeige4.1-3B (Nanbeige) — 0.0 (2026-02-11) MiniMax-M2.5 (MiniMax) — 34.8 (2026-02-12) Qwen3.5 397B A17B (Reasoning) (Alibaba) — 40.9 (2026-02-16) Qwen3.5 397B A17B (Non-reasoning) (Alibaba) — 35.6 (2026-02-16) Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) (Anthropic) — 53.0 (2026-02-17) Claude Sonnet 4.6 (Non-reasoning, High Effort) (Anthropic) — 46.2 (2026-02-17) Claude Sonnet 4.6 (Non-reasoning, Low Effort) (Anthropic) — 42.4 (2026-02-17) Tiny Aya Global (Cohere) — 0.0 (2026-02-17) Gemini 3.1 Pro Preview (Google) — 53.8 (2026-02-19) Mercury 2 (Inception) — 26.5 (2026-02-20) Qwen3.5 27B (Reasoning) (Alibaba) — 32.6 (2026-02-24) Qwen3.5 27B (Non-reasoning) (Alibaba) — 31.8 (2026-02-24) Qwen3.5 122B A10B (Reasoning) (Alibaba) — 31.1 (2026-02-24) Qwen3.5 122B A10B (Non-reasoning) (Alibaba) — 29.5 (2026-02-24) Qwen3.5 35B A3B (Reasoning) (Alibaba) — 26.5 (2026-02-24) Qwen3.5 35B A3B (Non-reasoning) (Alibaba) — 10.6 (2026-02-24) LFM2 24B A2B (Liquid AI) — 0.0 (2026-02-25) Qwen3.5 9B (Reasoning) (Alibaba) — 24.2 (2026-03-02) Qwen3.5 9B (Non-reasoning) (Alibaba) — 18.2 (2026-03-02) Qwen3.5 4B (Reasoning) (Alibaba) — 18.2 (2026-03-02) Qwen3.5 4B (Non-reasoning) (Alibaba) — 11.4 (2026-03-02) Qwen3.5 2B (Reasoning) (Alibaba) — 3.8 (2026-03-02) Qwen3.5 2B (Non-reasoning) (Alibaba) — 3.8 (2026-03-02) Qwen3.5 0.8B (Reasoning) (Alibaba) — 0.0 (2026-03-02) Qwen3.5 0.8B (Non-reasoning) (Alibaba) — 0.0 (2026-03-02) Gemini 3.1 Flash-Lite (Google) — 24.2 (2026-03-03) GPT-5.4 (xhigh) (OpenAI) — 57.6 (2026-03-05) GPT-5.4 (low) (OpenAI) — 43.2 (2026-03-05) GPT-5.4 (Non-reasoning) (OpenAI) — 37.9 (2026-03-05) Sarvam 30B (high) (Sarvam) — 2.3 (2026-03-06) Sarvam 105B (high) (Sarvam) — 1.5 (2026-03-06) Grok 4.20 0309 (Reasoning) (SpaceXAI) — 40.9 (2026-03-10) Grok 4.20 0309 (Non-reasoning) (SpaceXAI) — 22.0 (2026-03-10) Nemotron 3 Super 120B A12B (Reasoning) (NVIDIA) — 28.8 (2026-03-11) GLM-5-Turbo (Z AI) — 33.3 (2026-03-15) Mistral Small 4 (Reasoning) (Mistral) — 17.4 (2026-03-16) Mistral Small 4 (Non-reasoning) (Mistral) — 10.6 (2026-03-16) NVIDIA Nemotron 3 Nano 4B (NVIDIA) — 6.8 (2026-03-16) GPT-5.4 mini (xhigh) (OpenAI) — 52.3 (2026-03-17) GPT-5.4 nano (xhigh) (OpenAI) — 42.4 (2026-03-17) GPT-5.4 mini (medium) (OpenAI) — 34.1 (2026-03-17) GPT-5.4 nano (medium) (OpenAI) — 33.3 (2026-03-17) GPT-5.4 nano (Non-Reasoning) (OpenAI) — 24.2 (2026-03-17) GPT-5.4 mini (Non-Reasoning) (OpenAI) — 18.2 (2026-03-17) MiMo-V2-Pro (Xiaomi) — 40.9 (2026-03-18) MiniMax-M2.7 (MiniMax) — 39.4 (2026-03-18) MiMo-V2-Omni (Xiaomi) — 34.8 (2026-03-19) Nemotron Cascade 2 30B A3B (NVIDIA) — 21.2 (2026-03-19) KAT Coder Pro V2 (KwaiKAT) — 49.2 (2026-03-27) MiMo-V2-Omni-0327 (Xiaomi) — 35.6 (2026-03-27) Qwen3.5 Omni Plus (Alibaba) — 21.2 (2026-03-30) Qwen3.5 Omni Flash (Alibaba) — 8.3 (2026-03-30) GLM 5V Turbo (Reasoning) (Z AI) — 32.6 (2026-04-01) Trinity Large Thinking (Arcee AI) — 22.7 (2026-04-01) Qwen3.6 Plus (Alibaba) — 43.9 (2026-04-02) Gemma 4 31B (Reasoning) (Google) — 36.4 (2026-04-02) Step 3.5 Flash 2603 (StepFun) — 32.6 (2026-04-02) Gemma 4 31B (Non-reasoning) (Google) — 30.3 (2026-04-02) Gemma 4 26B A4B (Non-reasoning) (Google) — 25.0 (2026-04-02) Gemma 4 26B A4B (Reasoning) (Google) — 13.6 (2026-04-02) Gemma 4 E2B (Reasoning) (Google) — 3.0 (2026-04-02) Gemma 4 E2B (Non-reasoning) (Google) — 2.3 (2026-04-02) Gemma 4 E4B (Reasoning) (Google) — 8.3 (2026-04-03) Gemma 4 E4B (Non-reasoning) (Google) — 7.6 (2026-04-03) Solar Pro 3 (Upstage) — 7.6 (2026-04-06) GLM-5.1 (Reasoning) (Z AI) — 43.2 (2026-04-07) Grok 4.20 0309 v2 (Reasoning) (SpaceXAI) — 37.9 (2026-04-07) GLM-5.1 (Non-reasoning) (Z AI) — 35.6 (2026-04-07) Grok 4.20 0309 v2 (Non-reasoning) (SpaceXAI) — 16.7 (2026-04-07) Muse Spark (Meta) — 45.5 (2026-04-08) EXAONE 4.5 33B (LG AI Research) — 20.5 (2026-04-09) JT-MINI (China Mobile) — 18.2 (2026-04-15) Claude Opus 4.7 (Non-reasoning, High Effort) (Anthropic) — 54.5 (2026-04-16) Claude Opus 4.7 (Adaptive Reasoning, Max Effort) (Anthropic) — 51.5 (2026-04-16) Qwen3.6 35B A3B (Reasoning) (Alibaba) — 34.8 (2026-04-16) Qwen3.6 35B A3B (Non-reasoning) (Alibaba) — 25.8 (2026-04-16) Qwen3.6 Max Preview (Alibaba) — 43.9 (2026-04-20) Kimi K2.6 (Kimi) — 43.9 (2026-04-20) Kimi K2.6 (Non-reasoning) (Kimi) — 37.9 (2026-04-20) Ling 2.6 Flash (InclusionAI) — 21.2 (2026-04-21) MiMo-V2.5-Pro (Xiaomi) — 43.2 (2026-04-22) MiMo-V2.5 (Xiaomi) — 41.7 (2026-04-22) MiMo-V2.5-Pro (Non-reasoning) (Xiaomi) — 35.6 (2026-04-22) Qwen3.6 27B (Reasoning) (Alibaba) — 34.8 (2026-04-22) Qwen3.6 27B (Non-reasoning) (Alibaba) — 21.2 (2026-04-22) GPT-5.5 (xhigh) (OpenAI) — 60.6 (2026-04-23) GPT-5.5 (high) (OpenAI) — 59.8 (2026-04-23) GPT-5.5 (medium) (OpenAI) — 57.6 (2026-04-23) GPT-5.5 (low) (OpenAI) — 52.3 (2026-04-23) GPT-5.5 (Non-reasoning) (OpenAI) — 49.2 (2026-04-23) Hy3-preview (Reasoning) (Tencent) — 34.1 (2026-04-23) Hy3-preview (Non-reasoning) (Tencent) — 31.8 (2026-04-23) Ling-2.6-1T (InclusionAI) — 31.1 (2026-04-23) DeepSeek V4 Pro (Reasoning, Max Effort) (DeepSeek) — 46.2 (2026-04-24) DeepSeek V4 Pro (Reasoning, High Effort) (DeepSeek) — 41.7 (2026-04-24) DeepSeek V4 Flash (Reasoning, High Effort) (DeepSeek) — 38.6 (2026-04-24) DeepSeek V4 Pro (Non-reasoning) (DeepSeek) — 36.4 (2026-04-24) DeepSeek V4 Flash (Reasoning, Max Effort) (DeepSeek) — 35.6 (2026-04-24) DeepSeek V4 Flash (Non-reasoning) (DeepSeek) — 34.1 (2026-04-24) Mistral Medium 3.5 (Mistral) — 33.3 (2026-04-29) Nemotron 3 Nano Omni 30B A3B Reasoning (NVIDIA) — 8.3 (2026-04-29) Granite 4.1 30B (IBM) — 2.3 (2026-04-29) Granite 4.1 3B (IBM) — 2.3 (2026-04-29) Granite 4.1 8B (IBM) — 0.0 (2026-04-29) Grok 4.3 (high) (SpaceXAI) — 37.9 (2026-04-30) Grok 4.3 (medium) (SpaceXAI) — 30.3 (2026-04-30) Grok 4.3 (low) (SpaceXAI) — 26.5 (2026-04-30) Grok 4.3 (Non-reasoning) (SpaceXAI) — 18.9 (2026-04-30) GPT-5.5 Instant (May 2026) (OpenAI) — 42.4 (2026-05-05) Ring-2.6-1T (InclusionAI) — 28.8 (2026-05-08) MiniCPM-V 4.6 1.3B (OpenBMB) — 0.0 (2026-05-11) JT-35B-Flash (China Mobile) — 28.8 (2026-05-14) Qwen3.7 Max (Alibaba) — 50.8 (2026-05-19) Gemini 3.5 Flash (minimal) (Google) — 46.2 (2026-05-19) Gemini 3.5 Flash (high) (Google) — 40.9 (2026-05-19) Gemini 3.5 Flash (medium) (Google) — 39.4 (2026-05-19) Command A+ (Cohere) — 25.0 (2026-05-20) MiniCPM5-1B (Reasoning) (OpenBMB) — 0.0 (2026-05-25) MiniCPM5-1B (Non-reasoning) (OpenBMB) — 0.0 (2026-05-25) HyperNova 60B 2605 (high, based on gpt-oss-120b) (Multiverse Computing) — 23.5 (2026-05-26) Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (Anthropic) — 58.3 (2026-05-28) LFM2.5-8B-A1B (Liquid AI) — 4.5 (2026-05-28) Step 3.7 Flash (StepFun) — 35.6 (2026-05-29) Qwen3.7 Plus (Alibaba) — 47.0 (2026-06-01) MiniMax-M3 (MiniMax) — 42.4 (2026-06-01) Gemma 4 12B (Reasoning) (Google) — 18.2 (2026-06-03) Gemma 4 12B (Non-reasoning) (Google) — 11.4 (2026-06-03) Nemotron 3 Ultra 550B A55B (Reasoning) (NVIDIA) — 36.4 (2026-06-04) Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (Anthropic) — 62.9 (2026-06-09) North Mini Code (Cohere) — 31.1 (2026-06-09) Kimi K2.7 Code (Kimi) — 44.7 (2026-06-12) GLM-5.2 (max) (Z AI) — 50.8 (2026-06-16) GPT-5.6 Sol (max) (OpenAI) — 65.9 (2026-07-09) GPT-5.6 Sol (medium) (OpenAI) — 62.9 (2026-07-09) GPT-5.6 Terra (xhigh) (OpenAI) — 62.9 (2026-07-09) GPT-5.6 Sol (high) (OpenAI) — 62.1 (2026-07-09) GPT-5.6 Sol (xhigh) (OpenAI) — 61.4 (2026-07-09) GPT-5.6 Sol (low) (OpenAI) — 60.6 (2026-07-09) GPT-5.6 Terra (max) (OpenAI) — 57.6 (2026-07-09) GPT-5.6 Terra (high) (OpenAI) — 57.6 (2026-07-09) GPT-5.6 Terra (low) (OpenAI) — 43.9 (2026-07-09) Phi-4 Mini Instruct (Microsoft) — 0.0 (2024-02-26) Claude 3 Haiku (Anthropic) — 0.8 (2024-03-04) Llama 3.1 Instruct 405B (Meta) — 6.8 (2024-07-23) GPT-4o (Aug '24) (OpenAI) — 8.3 (2024-08-06) o1 (OpenAI) — 12.9 (2024-12-05) Grok 3 mini Reasoning (high) (SpaceXAI) — 17.4 (2025-02-19) Claude 3.7 Sonnet (Reasoning) (Anthropic) — 21.2 (2025-02-24) o3 (OpenAI) — 37.1 (2025-04-16) Grok 4 (SpaceXAI) — 37.9 (2025-07-10) GPT-5.1 (high) (OpenAI) — 45.5 (2025-11-13) Claude Opus 4.5 (Reasoning) (Anthropic) — 47.0 (2025-11-24) GPT-5.3 Codex (xhigh) (OpenAI) — 53.0 (2026-02-05) Gemini 3.1 Pro Preview (Google) — 53.8 (2026-02-19) GPT-5.4 (xhigh) (OpenAI) — 57.6 (2026-03-05) GPT-5.5 (xhigh) (OpenAI) — 60.6 (2026-04-23) Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (Anthropic) — 62.9 (2026-06-09) GPT-5.6 Sol (max) (OpenAI) — 65.9 (2026-07-09)
RankModel Accuracy PaperCodeYear
1 GPT-5.6 Sol (max) (OpenAI) 외부 65.9 artificialanalysis.ai 2026
2 Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (Anthropic) 외부 62.9 artificialanalysis.ai 2026
2 GPT-5.6 Sol (medium) (OpenAI) 외부 62.9 artificialanalysis.ai 2026
2 GPT-5.6 Terra (xhigh) (OpenAI) 외부 62.9 artificialanalysis.ai 2026
5 GPT-5.6 Sol (high) (OpenAI) 외부 62.1 artificialanalysis.ai 2026
6 GPT-5.6 Sol (xhigh) (OpenAI) 외부 61.4 artificialanalysis.ai 2026
7 GPT-5.5 (xhigh) (OpenAI) 외부 60.6 artificialanalysis.ai 2026
7 GPT-5.6 Sol (low) (OpenAI) 외부 60.6 artificialanalysis.ai 2026
9 GPT-5.5 (high) (OpenAI) 외부 59.8 artificialanalysis.ai 2026
10 Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (Anthropic) 외부 58.3 artificialanalysis.ai 2026
11 GPT-5.6 Terra (max) (OpenAI) 외부 57.6 artificialanalysis.ai 2026
11 GPT-5.4 (xhigh) (OpenAI) 외부 57.6 artificialanalysis.ai 2026
11 GPT-5.6 Terra (high) (OpenAI) 외부 57.6 artificialanalysis.ai 2026
11 GPT-5.5 (medium) (OpenAI) 외부 57.6 artificialanalysis.ai 2026
15 Claude Opus 4.7 (Non-reasoning, High Effort) (Anthropic) 외부 54.5 artificialanalysis.ai 2026
16 Gemini 3.1 Pro Preview (Google) 외부 53.8 artificialanalysis.ai 2026
17 GPT-5.3 Codex (xhigh) (OpenAI) 외부 53 artificialanalysis.ai 2026
17 Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) (Anthropic) 외부 53 artificialanalysis.ai 2026
19 GPT-5.5 (low) (OpenAI) 외부 52.3 artificialanalysis.ai 2026
19 GPT-5.4 mini (xhigh) (OpenAI) 외부 52.3 artificialanalysis.ai 2026
1–20 / 432 다음 → 페이지당 10 20 50 100