paper-with-me

Code Generation 벤치마크

Code Generation on LiveCodeBench

350개 결과 · ⬇ CSV · JSON

Pass@1

0.2 23.07 45.95 68.83 91.7 2023-03 2026-11 Claude Instant (Anthropic) — 10.9 (2023-03-14) Claude 2.0 (Anthropic) — 17.1 (2023-07-11) Llama 2 Chat 70B (Meta) — 9.8 (2023-07-18) Llama 2 Chat 13B (Meta) — 9.8 (2023-07-18) Llama 2 Chat 7B (Meta) — 0.2 (2023-07-18) Mistral 7B Instruct (Mistral) — 4.6 (2023-09-27) GPT-4 Turbo (OpenAI) — 29.1 (2023-11-06) Claude 2.1 (Anthropic) — 19.5 (2023-11-21) Gemini 1.0 Pro (Google) — 11.6 (2023-12-06) Mistral Medium (Mistral) — 9.9 (2023-12-11) Mixtral 8x7B Instruct (Mistral) — 6.6 (2023-12-11) OpenChat 3.5 (1210) (OpenChat) — 11.5 (2023-12-18) Mistral Large (Feb '24) (Mistral) — 17.8 (2024-02-26) Phi-4 Mini Instruct (Microsoft) — 12.6 (2024-02-26) Mistral Small (Feb '24) (Mistral) — 11.1 (2024-02-26) Claude 3 Opus (Anthropic) — 27.9 (2024-03-04) Claude 3 Sonnet (Anthropic) — 17.5 (2024-03-04) Claude 3 Haiku (Anthropic) — 15.4 (2024-03-04) Command-R (Mar '24) (Cohere) — 4.8 (2024-03-12) DBRX Instruct (Databricks) — 9.3 (2024-03-27) Command-R+ (Apr '24) (Cohere) — 12.2 (2024-04-04) Mixtral 8x22B Instruct (Mistral) — 14.8 (2024-04-17) Llama 3 Instruct 70B (Meta) — 19.8 (2024-04-18) Llama 3 Instruct 8B (Meta) — 9.6 (2024-04-18) Phi-3 Mini Instruct 3.8B (Microsoft) — 11.6 (2024-04-23) GPT-4o (May '24) (OpenAI) — 33.4 (2024-05-13) Gemini 1.5 Flash (May '24) (Google) — 19.6 (2024-05-14) Gemini 1.5 Pro (May '24) (Google) — 24.4 (2024-05-15) Qwen2 Instruct 72B (Alibaba) — 15.9 (2024-06-07) DeepSeek Coder V2 Lite Instruct (DeepSeek) — 15.8 (2024-06-17) GPT-4o mini (OpenAI) — 23.4 (2024-07-18) Llama 3.1 Instruct 405B (Meta) — 30.5 (2024-07-23) Llama 3.1 Instruct 70B (Meta) — 23.2 (2024-07-23) Llama 3.1 Instruct 8B (Meta) — 11.6 (2024-07-23) Mistral Large 2 (Jul '24) (Mistral) — 26.7 (2024-07-24) GPT-4o (Aug '24) (OpenAI) — 31.7 (2024-08-06) Grok Beta (SpaceXAI) — 24.1 (2024-08-13) Hermes 3 - Llama-3.1 70B (Nous Research) — 18.8 (2024-08-15) Jamba 1.5 Large (AI21 Labs) — 14.3 (2024-08-22) Jamba 1.5 Mini (AI21 Labs) — 6.2 (2024-08-22) o1-mini (OpenAI) — 57.6 (2024-09-12) Mistral Small (Sep '24) (Mistral) — 14.1 (2024-09-17) Qwen2.5 Instruct 72B (Alibaba) — 27.6 (2024-09-19) Qwen2.5 Instruct 32B (Alibaba) — 24.8 (2024-09-19) Qwen2.5 Coder Instruct 7B (Alibaba) — 12.6 (2024-09-19) Gemini 1.5 Pro (Sep '24) (Google) — 31.6 (2024-09-24) Gemini 1.5 Flash (Sep '24) (Google) — 27.3 (2024-09-24) Llama 3.2 Instruct 90B (Vision) (Meta) — 21.4 (2024-09-25) Llama 3.2 Instruct 11B (Vision) (Meta) — 11.0 (2024-09-25) Llama 3.2 Instruct 3B (Meta) — 8.3 (2024-09-25) Molmo 7B-D (Allen Institute for AI) — 3.9 (2024-09-25) Llama 3.2 Instruct 1B (Meta) — 1.9 (2024-09-25) LFM 40B (Liquid AI) — 9.6 (2024-09-30) Gemini 1.5 Flash-8B (Google) — 21.7 (2024-10-03) Llama 3.1 Nemotron Instruct 70B (NVIDIA) — 16.9 (2024-10-15) Claude 3.5 Sonnet (Oct '24) (Anthropic) — 38.1 (2024-10-22) Claude 3.5 Haiku (Anthropic) — 31.4 (2024-10-22) Qwen2.5 Coder Instruct 32B (Alibaba) — 29.5 (2024-11-11) Mistral Large 2 (Nov '24) (Mistral) — 29.3 (2024-11-18) Pixtral Large (Mistral) — 26.1 (2024-11-18) Qwen2.5 Turbo (Alibaba) — 16.3 (2024-11-18) GPT-4o (Nov '24) (OpenAI) — 30.9 (2024-11-20) OLMo 2 7B (Allen Institute for AI) — 4.1 (2024-11-26) QwQ 32B-Preview (Alibaba) — 33.7 (2024-11-27) Nova Pro (Amazon) — 23.3 (2024-12-03) Nova Lite (Amazon) — 16.7 (2024-12-03) Nova Micro (Amazon) — 14.0 (2024-12-03) o1 (OpenAI) — 67.9 (2024-12-05) Llama 3.3 Instruct 70B (Meta) — 28.8 (2024-12-06) Gemini 2.0 Flash (experimental) (Google) — 21.0 (2024-12-11) Grok 2 (Dec '24) (SpaceXAI) — 26.7 (2024-12-12) Phi-4 (Microsoft) — 23.1 (2024-12-12) DeepSeek V3 (Dec '24) (DeepSeek) — 35.9 (2024-12-26) DeepSeek R1 (Jan '25) (DeepSeek) — 61.7 (2025-01-20) DeepSeek R1 Distill Qwen 14B (DeepSeek) — 37.6 (2025-01-20) DeepSeek R1 Distill Qwen 32B (DeepSeek) — 27.0 (2025-01-20) DeepSeek R1 Distill Llama 70B (DeepSeek) — 26.6 (2025-01-20) DeepSeek R1 Distill Llama 8B (DeepSeek) — 23.3 (2025-01-20) DeepSeek R1 Distill Qwen 1.5B (DeepSeek) — 7.0 (2025-01-20) Gemini 2.0 Flash Thinking Experimental (Jan '25) (Google) — 32.1 (2025-01-21) Sonar (Perplexity) — 29.5 (2025-01-21) Sonar Pro (Perplexity) — 27.5 (2025-01-21) Qwen2.5 Max (Alibaba) — 35.9 (2025-01-28) Llama 3.1 Tulu3 405B (Allen Institute for AI) — 29.1 (2025-01-30) Mistral Small 3 (Mistral) — 25.2 (2025-01-30) o3-mini (High) (OpenAI) — 73.4 (2025-01-31) o3-mini (OpenAI) — 71.7 (2025-01-31) Gemini 2.0 Pro Experimental (Feb '25) (Google) — 34.7 (2025-02-05) Gemini 2.0 Flash (Feb '25) (Google) — 33.4 (2025-02-05) Gemini 2.0 Flash-Lite (Preview) (Google) — 17.9 (2025-02-05) DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) (Nous Research) — 8.5 (2025-02-13) Grok 3 mini Reasoning (High) (SpaceXAI) — 69.6 (2025-02-19) Grok 3 (SpaceXAI) — 42.5 (2025-02-19) Claude 3.7 Sonnet (Reasoning) (Anthropic) — 47.3 (2025-02-24) Claude 3.7 Sonnet (Non-reasoning) (Anthropic) — 39.4 (2025-02-24) Gemini 2.0 Flash-Lite (Feb '25) (Google) — 18.5 (2025-02-25) Phi-4 Multimodal Instruct (Microsoft) — 13.1 (2025-02-26) QwQ 32B (Alibaba) — 63.1 (2025-03-05) Jamba 1.6 Large (AI21 Labs) — 17.2 (2025-03-06) Jamba 1.6 Mini (AI21 Labs) — 7.1 (2025-03-06) Reka Flash 3 (Reka AI) — 43.5 (2025-03-10) Gemma 3 27B Instruct (Google) — 13.7 (2025-03-12) Gemma 3 12B Instruct (Google) — 13.7 (2025-03-12) Gemma 3 4B Instruct (Google) — 11.2 (2025-03-12) Command A (Cohere) — 28.7 (2025-03-13) DeepHermes 3 - Mistral 24B Preview (Non-reasoning) (Nous Research) — 19.5 (2025-03-13) OLMo 2 32B (Allen Institute for AI) — 6.8 (2025-03-13) Gemma 3 1B Instruct (Google) — 1.7 (2025-03-13) Mistral Small 3.1 (Mistral) — 21.2 (2025-03-17) Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) (NVIDIA) — 28.0 (2025-03-18) Llama 3.3 Nemotron Super 49B v1 (Reasoning) (NVIDIA) — 27.7 (2025-03-18) Gemini 2.5 Pro Preview (Mar' 25) (Google) — 77.8 (2025-03-25) DeepSeek V3 0324 (DeepSeek) — 40.5 (2025-03-25) GPT-4o (March 2025, chatgpt-4o-latest) (OpenAI) — 42.5 (2025-03-27) Llama 4 Maverick (Meta) — 39.7 (2025-04-05) Llama 4 Scout (Meta) — 29.9 (2025-04-05) Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) (NVIDIA) — 64.1 (2025-04-07) GPT-4.1 mini (OpenAI) — 48.3 (2025-04-14) GPT-4.1 (OpenAI) — 45.7 (2025-04-14) GPT-4.1 nano (OpenAI) — 32.6 (2025-04-14) o4-mini (High) (OpenAI) — 85.9 (2025-04-16) o3 (OpenAI) — 80.8 (2025-04-16) Granite 3.3 8B (Non-reasoning) (IBM) — 12.7 (2025-04-16) Gemini 2.5 Flash Preview (Reasoning) (Google) — 50.5 (2025-04-17) Gemini 2.5 Flash Preview (Non-reasoning) (Google) — 40.6 (2025-04-17) Qwen3 235B A22B (Reasoning) (Alibaba) — 62.2 (2025-04-28) Qwen3 32B (Reasoning) (Alibaba) — 54.6 (2025-04-28) Qwen3 14B (Reasoning) (Alibaba) — 52.3 (2025-04-28) Qwen3 30B A3B (Reasoning) (Alibaba) — 50.6 (2025-04-28) Qwen3 4B (Reasoning) (Alibaba) — 46.5 (2025-04-28) Qwen3 8B (Reasoning) (Alibaba) — 40.6 (2025-04-28) Qwen3 235B A22B (Non-reasoning) (Alibaba) — 34.3 (2025-04-28) Qwen3 30B A3B (Non-reasoning) (Alibaba) — 32.2 (2025-04-28) Qwen3 1.7B (Reasoning) (Alibaba) — 30.8 (2025-04-28) Qwen3 32B (Non-reasoning) (Alibaba) — 28.8 (2025-04-28) Qwen3 14B (Non-reasoning) (Alibaba) — 28.0 (2025-04-28) Qwen3 4B (Non-reasoning) (Alibaba) — 23.3 (2025-04-28) Qwen3 8B (Non-reasoning) (Alibaba) — 20.2 (2025-04-28) Qwen3 1.7B (Non-reasoning) (Alibaba) — 12.6 (2025-04-28) Qwen3 0.6B (Reasoning) (Alibaba) — 12.1 (2025-04-28) Qwen3 0.6B (Non-reasoning) (Alibaba) — 7.3 (2025-04-28) Nova Premier (Amazon) — 31.7 (2025-04-30) Gemini 2.5 Pro Preview (May' 25) (Google) — 77.0 (2025-05-06) Mistral Medium 3 (Mistral) — 40.0 (2025-05-07) Gemini 2.5 Flash (Reasoning) (Google) — 69.5 (2025-05-20) Gemini 2.5 Flash (Non-reasoning) (Google) — 49.5 (2025-05-20) Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) (NVIDIA) — 49.3 (2025-05-20) Solar Pro 2 (Preview) (Reasoning) (Upstage) — 46.2 (2025-05-20) Solar Pro 2 (Preview) (Non-reasoning) (Upstage) — 38.5 (2025-05-20) Gemma 3n E4B Instruct Preview (May '25) (Google) — 13.8 (2025-05-20) Devstral Small (May '25) (Mistral) — 25.8 (2025-05-21) Claude 4 Sonnet (Reasoning) (Anthropic) — 65.5 (2025-05-22) Claude 4 Opus (Reasoning) (Anthropic) — 63.6 (2025-05-22) Claude 4 Opus (Non-reasoning) (Anthropic) — 54.2 (2025-05-22) Claude 4 Sonnet (Non-reasoning) (Anthropic) — 44.9 (2025-05-22) Sarvam M (Reasoning, Based on Mistral Small 3.1) (Sarvam) — 29.5 (2025-05-23) DeepSeek R1 0528 (May '25) (DeepSeek) — 77.0 (2025-05-28) DeepSeek R1 0528 Qwen3 8B (DeepSeek) — 51.3 (2025-05-29) Gemini 2.5 Pro (Google) — 80.1 (2025-06-05) Magistral Medium 1 (Mistral) — 52.7 (2025-06-10) Magistral Small 1 (Mistral) — 51.4 (2025-06-10) MiniMax M1 80k (MiniMax) — 71.1 (2025-06-17) MiniMax M1 40k (MiniMax) — 65.7 (2025-06-17) Gemini 2.5 Flash-Lite (Reasoning) (Google) — 59.3 (2025-06-17) Gemini 2.5 Flash-Lite (Non-reasoning) (Google) — 40.0 (2025-06-17) Mistral Small 3.2 (Mistral) — 27.5 (2025-06-20) Gemma 3n E4B Instruct (Google) — 14.6 (2025-06-26) Gemma 3n E2B Instruct (Google) — 9.5 (2025-06-26) ERNIE 4.5 300B A47B (Baidu) — 46.7 (2025-06-30) Jamba 1.7 Large (AI21 Labs) — 18.1 (2025-07-07) Jamba 1.7 Mini (AI21 Labs) — 6.1 (2025-07-07) Solar Pro 2 (Reasoning) (Upstage) — 61.6 (2025-07-09) Solar Pro 2 (Non-reasoning) (Upstage) — 42.4 (2025-07-09) Grok 4 (SpaceXAI) — 81.9 (2025-07-10) Devstral Medium (Mistral) — 33.7 (2025-07-10) Devstral Small (Jul '25) (Mistral) — 25.4 (2025-07-10) LFM2 1.2B (Liquid AI) — 2.0 (2025-07-10) Kimi K2 (Kimi) — 55.6 (2025-07-11) EXAONE 4.0 32B (Reasoning) (LG AI Research) — 74.7 (2025-07-15) Exaone 4.0 1.2B (Reasoning) (LG AI Research) — 51.6 (2025-07-15) EXAONE 4.0 32B (Non-reasoning) (LG AI Research) — 47.2 (2025-07-15) Exaone 4.0 1.2B (Non-reasoning) (LG AI Research) — 29.3 (2025-07-15) Qwen3 235B A22B 2507 Instruct (Alibaba) — 52.4 (2025-07-21) Qwen3 Coder 480B A35B Instruct (Alibaba) — 58.5 (2025-07-22) Qwen3 235B A22B 2507 (Reasoning) (Alibaba) — 78.8 (2025-07-25) Llama Nemotron Super 49B v1.5 (Reasoning) (NVIDIA) — 73.7 (2025-07-25) Llama Nemotron Super 49B v1.5 (Non-reasoning) (NVIDIA) — 29.0 (2025-07-25) GLM-4.5 (Reasoning) (Z AI) — 73.8 (2025-07-28) GLM-4.5-Air (Z AI) — 68.4 (2025-07-28) Qwen3 30B A3B 2507 Instruct (Alibaba) — 51.5 (2025-07-29) Qwen3 30B A3B 2507 (Reasoning) (Alibaba) — 70.7 (2025-07-30) Qwen3 Coder 30B A3B Instruct (Alibaba) — 40.3 (2025-07-31) gpt-oss-120b (High) (OpenAI) — 87.8 (2025-08-05) gpt-oss-20b (High) (OpenAI) — 77.7 (2025-08-05) gpt-oss-120b (Low) (OpenAI) — 70.7 (2025-08-05) Claude 4.1 Opus (Reasoning) (Anthropic) — 65.4 (2025-08-05) gpt-oss-20b (Low) (OpenAI) — 65.2 (2025-08-05) Qwen3 4B 2507 (Reasoning) (Alibaba) — 64.1 (2025-08-06) Qwen3 4B 2507 Instruct (Alibaba) — 37.7 (2025-08-06) GPT-5 (High) (OpenAI) — 84.6 (2025-08-07) GPT-5 mini (High) (OpenAI) — 83.8 (2025-08-07) GPT-5 nano (High) (OpenAI) — 78.9 (2025-08-07) GPT-5 (Low) (OpenAI) — 76.3 (2025-08-07) GPT-5 nano (Medium) (OpenAI) — 76.3 (2025-08-07) GPT-5 (Medium) (OpenAI) — 70.3 (2025-08-07) GPT-5 mini (Medium) (OpenAI) — 69.2 (2025-08-07) GPT-5 (Minimal) (OpenAI) — 55.8 (2025-08-07) GPT-5 mini (Minimal) (OpenAI) — 54.5 (2025-08-07) GPT-5 (ChatGPT) (OpenAI) — 54.3 (2025-08-07) GPT-5 nano (Minimal) (OpenAI) — 47.0 (2025-08-07) GLM-4.5V (Reasoning) (Z AI) — 60.4 (2025-08-11) GLM-4.5V (Non-reasoning) (Z AI) — 35.2 (2025-08-11) Mistral Medium 3.1 (Mistral) — 40.6 (2025-08-12) Gemma 3 270M (Google) — 0.3 (2025-08-14) NVIDIA Nemotron Nano 9B V2 (Reasoning) (NVIDIA) — 72.4 (2025-08-18) NVIDIA Nemotron Nano 9B V2 (Non-reasoning) (NVIDIA) — 70.1 (2025-08-18) Seed-OSS-36B-Instruct (ByteDance Seed) — 76.5 (2025-08-20) DeepSeek V3.1 (Reasoning) (DeepSeek) — 78.4 (2025-08-21) DeepSeek V3.1 (Non-reasoning) (DeepSeek) — 57.7 (2025-08-21) Hermes 4 - Llama-3.1 405B (Reasoning) (Nous Research) — 68.6 (2025-08-27) Hermes 4 - Llama-3.1 70B (Reasoning) (Nous Research) — 65.3 (2025-08-27) Hermes 4 - Llama-3.1 405B (Non-reasoning) (Nous Research) — 54.6 (2025-08-27) Hermes 4 - Llama-3.1 70B (Non-reasoning) (Nous Research) — 26.9 (2025-08-27) Grok Code Fast 1 (SpaceXAI) — 65.7 (2025-08-28) Dream-Coder — 21.4 (2025-09-01) Qwen3 Max (Preview) (Alibaba) — 65.1 (2025-09-05) Kimi K2 0905 (Kimi) — 61.0 (2025-09-05) Ling-mini-2.0 (InclusionAI) — 42.9 (2025-09-09) Qwen3 Next 80B A3B (Reasoning) (Alibaba) — 78.4 (2025-09-11) Qwen3 Next 80B A3B Instruct (Alibaba) — 68.4 (2025-09-11) Magistral Small 1.2 (Mistral) — 72.3 (2025-09-17) Ling-flash-2.0 (InclusionAI) — 58.9 (2025-09-17) Magistral Medium 1.2 (Mistral) — 75.0 (2025-09-18) Grok 4 Fast (Reasoning) (SpaceXAI) — 83.2 (2025-09-19) Ring-flash-2.0 (InclusionAI) — 62.8 (2025-09-19) Grok 4 Fast (Non-reasoning) (SpaceXAI) — 40.1 (2025-09-19) DeepSeek V3.1 Terminus (Reasoning) (DeepSeek) — 79.8 (2025-09-22) Qwen3 Omni 30B A3B (Reasoning) (Alibaba) — 67.9 (2025-09-22) DeepSeek V3.1 Terminus (Non-reasoning) (DeepSeek) — 52.9 (2025-09-22) Qwen3 Omni 30B A3B Instruct (Alibaba) — 42.2 (2025-09-22) Granite 4.0 H Small (IBM) — 25.1 (2025-09-22) Granite 4.0 Micro (IBM) — 18.0 (2025-09-22) GPT-5 Codex (High) (OpenAI) — 84.0 (2025-09-23) Qwen3 Max (Alibaba) — 76.7 (2025-09-23) Qwen3 VL 235B A22B (Reasoning) (Alibaba) — 64.6 (2025-09-23) Qwen3 VL 235B A22B Instruct (Alibaba) — 59.4 (2025-09-23) LFM2 2.6B (Liquid AI) — 8.1 (2025-09-23) Gemini 2.5 Flash Preview (Sep '25) (Reasoning) (Google) — 71.3 (2025-09-25) Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning) (Google) — 68.8 (2025-09-25) Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning) (Google) — 64.1 (2025-09-25) Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning) (Google) — 62.5 (2025-09-25) DeepSeek V3.2 Exp (Reasoning) (DeepSeek) — 78.9 (2025-09-29) Claude 4.5 Sonnet (Reasoning) (Anthropic) — 71.4 (2025-09-29) Claude 4.5 Sonnet (Non-reasoning) (Anthropic) — 59.0 (2025-09-29) DeepSeek V3.2 Exp (Non-reasoning) (DeepSeek) — 55.4 (2025-09-29) Apriel-v1.5-15B-Thinker (ServiceNow) — 72.8 (2025-09-30) GLM-4.6 (Reasoning) (Z AI) — 69.5 (2025-09-30) CWM — 68.6 (2025-09-30) GLM-4.6 (Non-reasoning) (Z AI) — 56.1 (2025-09-30) Qwen3 VL 30B A3B (Reasoning) (Alibaba) — 69.7 (2025-10-03) Qwen3 VL 30B A3B Instruct (Alibaba) — 47.6 (2025-10-03) LFM2 8B A1B (Liquid AI) — 15.1 (2025-10-07) Ling-1T (InclusionAI) — 67.7 (2025-10-08) Jamba Reasoning 3B (AI21 Labs) — 21.0 (2025-10-08) Ring-1T (InclusionAI) — 64.3 (2025-10-13) Qwen3 VL 8B (Reasoning) (Alibaba) — 35.3 (2025-10-14) Qwen3 VL 8B Instruct (Alibaba) — 33.2 (2025-10-14) Qwen3 VL 4B (Reasoning) (Alibaba) — 32.0 (2025-10-14) Qwen3 VL 4B Instruct (Alibaba) — 29.0 (2025-10-14) Claude 4.5 Haiku (Reasoning) (Anthropic) — 61.5 (2025-10-15) Claude 4.5 Haiku (Non-reasoning) (Anthropic) — 51.1 (2025-10-15) Qwen3 VL 32B (Reasoning) (Alibaba) — 73.8 (2025-10-21) Qwen3 VL 32B Instruct (Alibaba) — 51.4 (2025-10-21) MiniMax-M2 (MiniMax) — 82.6 (2025-10-26) NVIDIA Nemotron Nano 12B v2 VL (Reasoning) (NVIDIA) — 69.4 (2025-10-28) NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning) (NVIDIA) — 34.5 (2025-10-28) Granite 4.0 H 1B (IBM) — 11.5 (2025-10-28) Granite 4.0 1B (IBM) — 4.7 (2025-10-28) Granite 4.0 350M (IBM) — 2.4 (2025-10-28) Granite 4.0 H 350M (IBM) — 1.9 (2025-10-28) Nova 2.0 Lite (High) (Amazon) — 71.1 (2025-10-29) Nova 2.0 Lite (Medium) (Amazon) — 66.3 (2025-10-29) Nova 2.0 Lite (Low) (Amazon) — 46.9 (2025-10-29) Nova 2.0 Lite (Non-reasoning) (Amazon) — 34.6 (2025-10-29) Kimi Linear 48B A3B Instruct (Kimi) — 37.8 (2025-10-30) Qwen3 Max Thinking (Preview) (Alibaba) — 53.5 (2025-11-03) Kimi K2 Thinking (Kimi) — 85.3 (2025-11-06) Doubao Seed Code (ByteDance Seed) — 76.6 (2025-11-11) KAT-Coder-Pro V1 (KwaiKAT) — 74.7 (2025-11-11) GPT-5.1 (High) (OpenAI) — 86.8 (2025-11-13) GPT-5.1 Codex (High) (OpenAI) — 84.9 (2025-11-13) GPT-5.1 Codex mini (High) (OpenAI) — 83.6 (2025-11-13) ERNIE 5.0 Thinking Preview (Baidu) — 81.2 (2025-11-13) GPT-5.1 (Non-reasoning) (OpenAI) — 49.4 (2025-11-13) Gemini 3 Pro Preview (High) (Google) — 91.7 (2025-11-18) Gemini 3 Pro Preview (Low) (Google) — 85.7 (2025-11-18) Cogito v2.1 (Reasoning) (Deep Cogito) — 68.8 (2025-11-18) Grok 4.1 Fast (Reasoning) (SpaceXAI) — 82.2 (2025-11-19) Grok 4.1 Fast (Non-reasoning) (SpaceXAI) — 39.9 (2025-11-19) Olmo 3 32B Think (Allen Institute for AI) — 67.2 (2025-11-20) Olmo 3 7B Think (Allen Institute for AI) — 61.7 (2025-11-20) Olmo 3 7B Instruct (Allen Institute for AI) — 26.6 (2025-11-20) Claude Opus 4.5 (Reasoning) (Anthropic) — 87.1 (2025-11-24) Claude Opus 4.5 (Non-reasoning) (Anthropic) — 73.8 (2025-11-24) Apriel-v1.6-15B-Thinker (ServiceNow) — 80.7 (2025-11-25) Nova 2.0 Omni (Medium) (Amazon) — 66.0 (2025-11-26) Nova 2.0 Omni (Low) (Amazon) — 59.2 (2025-11-26) Nova 2.0 Omni (Non-reasoning) (Amazon) — 30.5 (2025-11-26) INTELLECT-3 (Based on GLM-4.5-Air) (Prime Intellect) — 77.7 (2025-11-27) Nova 2.0 Pro Preview (Medium) (Amazon) — 73.0 (2025-11-27) Nova 2.0 Pro Preview (Low) (Amazon) — 63.8 (2025-11-27) Nova 2.0 Pro Preview (Non-reasoning) (Amazon) — 47.3 (2025-11-27) DeepSeek V3.2 Speciale (DeepSeek) — 89.6 (2025-12-01) DeepSeek V3.2 (Reasoning) (DeepSeek) — 86.2 (2025-12-01) DeepSeek V3.2 (Non-reasoning) (DeepSeek) — 59.3 (2025-12-01) Mistral Large 3 (Mistral) — 46.5 (2025-12-02) Ministral 3 14B (Mistral) — 35.1 (2025-12-02) Ministral 3 8B (Mistral) — 30.3 (2025-12-02) Ministral 3 3B (Mistral) — 24.7 (2025-12-02) ThinkMerge — 8.28 (2025-12-02) Motif-2-12.7B-Reasoning (Motif Technologies) — 65.1 (2025-12-04) K2-V2 (High) (Institute of Foundation Models) — 69.4 (2025-12-05) K2-V2 (Medium) (Institute of Foundation Models) — 54.1 (2025-12-05) K2-V2 (Low) (Institute of Foundation Models) — 39.3 (2025-12-05) GLM-4.6V (Non-reasoning) (Z AI) — 41.1 (2025-12-08) GLM-4.6V (Reasoning) (Z AI) — 16.0 (2025-12-08) Devstral 2 (Mistral) — 44.8 (2025-12-09) Devstral Small 2 (Mistral) — 34.8 (2025-12-09) GPT-5.2 (Medium) (OpenAI) — 89.4 (2025-12-11) GPT-5.2 (Xhigh) (OpenAI) — 88.9 (2025-12-11) GPT-5.2 (Non-reasoning) (OpenAI) — 66.9 (2025-12-11) Mi:dm K 2.5 Pro (Korea Telecom) — 65.6 (2025-12-11) Mi:dm K 2.5 Pro Preview (Korea Telecom) — 57.6 (2025-12-11) Olmo 3.1 32B Think (Allen Institute for AI) — 69.5 (2025-12-12) NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) (NVIDIA) — 74.1 (2025-12-15) NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) (NVIDIA) — 36.0 (2025-12-15) MiMo-V2-Flash (Reasoning) (Xiaomi) — 86.8 (2025-12-16) MiMo-V2-Flash (Non-reasoning) (Xiaomi) — 40.2 (2025-12-16) Gemini 3 Flash Preview (Reasoning) (Google) — 90.8 (2025-12-17) DreamPRM-Code — 80.9 (2025-12-17) Gemini 3 Flash Preview (Non-reasoning) (Google) — 79.7 (2025-12-17) GLM-4.7 (Reasoning) (Z AI) — 89.4 (2025-12-22) GLM-4.7 (Non-reasoning) (Z AI) — 56.2 (2025-12-22) MiniMax-M2.1 (MiniMax) — 81.0 (2025-12-23) HyperCLOVA X SEED Think (32B) (Naver) — 62.9 (2025-12-26) K-EXAONE (Reasoning) (LG AI Research) — 76.8 (2025-12-31) Falcon-H1R-7B (TII UAE) — 72.4 (2026-01-04) Bridging Online and Offline RL: Contextu — 6.2 (2026-02-03) Embarrassingly Simple Self-Distillation — 55.3 (2026-04-01) DuST — 6.2 (2026-05-11) Claude Instant (Anthropic) — 10.9 (2023-03-14) Claude 2.0 (Anthropic) — 17.1 (2023-07-11) GPT-4 Turbo (OpenAI) — 29.1 (2023-11-06) GPT-4o (May '24) (OpenAI) — 33.4 (2024-05-13) o1-mini (OpenAI) — 57.6 (2024-09-12) o1 (OpenAI) — 67.9 (2024-12-05) o3-mini (High) (OpenAI) — 73.4 (2025-01-31) Gemini 2.5 Pro Preview (Mar' 25) (Google) — 77.8 (2025-03-25) o4-mini (High) (OpenAI) — 85.9 (2025-04-16) gpt-oss-120b (High) (OpenAI) — 87.8 (2025-08-05) Gemini 3 Pro Preview (High) (Google) — 91.7 (2025-11-18)
RankModel Pass@1 PaperCodeYear
21 MiniMax-M2 (MiniMax) 외부 82.6 artificialanalysis.ai 2025
22 Grok 4.1 Fast (Reasoning) (SpaceXAI) 외부 82.2 artificialanalysis.ai 2025
23 Grok 4 (SpaceXAI) 외부 81.9 artificialanalysis.ai 2025
24 ERNIE 5.0 Thinking Preview (Baidu) 외부 81.2 artificialanalysis.ai 2025
25 MiniMax-M2.1 (MiniMax) 외부 81 artificialanalysis.ai 2025
26 DreamPRM-Code 자동 추출 80.9 DreamPRM-Code: Function-as-Step Process Reward Model with Label Correction for LLM Coding 2025
27 o3 (OpenAI) 외부 80.8 artificialanalysis.ai 2025
28 Apriel-v1.6-15B-Thinker (ServiceNow) 외부 80.7 artificialanalysis.ai 2025
29 Gemini 2.5 Pro (Google) 외부 80.1 artificialanalysis.ai 2025
30 DeepSeek V3.1 Terminus (Reasoning) (DeepSeek) 외부 79.8 artificialanalysis.ai 2025
31 Gemini 3 Flash Preview (Non-reasoning) (Google) 외부 79.7 artificialanalysis.ai 2025
32 DeepSeek V3.2 Exp (Reasoning) (DeepSeek) 외부 78.9 artificialanalysis.ai 2025
32 GPT-5 nano (High) (OpenAI) 외부 78.9 artificialanalysis.ai 2025
34 Qwen3 235B A22B 2507 (Reasoning) (Alibaba) 외부 78.8 artificialanalysis.ai 2025
35 DeepSeek V3.1 (Reasoning) (DeepSeek) 외부 78.4 artificialanalysis.ai 2025
35 Qwen3 Next 80B A3B (Reasoning) (Alibaba) 외부 78.4 artificialanalysis.ai 2025
37 Gemini 2.5 Pro Preview (Mar' 25) (Google) 외부 77.8 artificialanalysis.ai 2025
38 INTELLECT-3 (Based on GLM-4.5-Air) (Prime Intellect) 외부 77.7 artificialanalysis.ai 2025
38 gpt-oss-20b (High) (OpenAI) 외부 77.7 artificialanalysis.ai 2025
40 Gemini 2.5 Pro Preview (May' 25) (Google) 외부 77 artificialanalysis.ai 2025
← 이전 21–40 / 350 다음 → 페이지당 10 20 50 100