paper-with-me

Code Generation 벤치마크

Code Generation on LiveCodeBench

350개 결과 · ⬇ CSV · JSON

Pass@1

0.2 23.07 45.95 68.83 91.7 2023-03 2026-09 Claude Instant (Anthropic) — 10.9 (2023-03-14) Claude 2.0 (Anthropic) — 17.1 (2023-07-11) Llama 2 Chat 70B (Meta) — 9.8 (2023-07-18) Llama 2 Chat 13B (Meta) — 9.8 (2023-07-18) Llama 2 Chat 7B (Meta) — 0.2 (2023-07-18) Mistral 7B Instruct (Mistral) — 4.6 (2023-09-27) GPT-4 Turbo (OpenAI) — 29.1 (2023-11-06) Claude 2.1 (Anthropic) — 19.5 (2023-11-21) Gemini 1.0 Pro (Google) — 11.6 (2023-12-06) Mistral Medium (Mistral) — 9.9 (2023-12-11) Mixtral 8x7B Instruct (Mistral) — 6.6 (2023-12-11) OpenChat 3.5 (1210) (OpenChat) — 11.5 (2023-12-18) Mistral Large (Feb '24) (Mistral) — 17.8 (2024-02-26) Phi-4 Mini Instruct (Microsoft) — 12.6 (2024-02-26) Mistral Small (Feb '24) (Mistral) — 11.1 (2024-02-26) Claude 3 Opus (Anthropic) — 27.9 (2024-03-04) Claude 3 Sonnet (Anthropic) — 17.5 (2024-03-04) Claude 3 Haiku (Anthropic) — 15.4 (2024-03-04) Command-R (Mar '24) (Cohere) — 4.8 (2024-03-12) DBRX Instruct (Databricks) — 9.3 (2024-03-27) Command-R+ (Apr '24) (Cohere) — 12.2 (2024-04-04) Mixtral 8x22B Instruct (Mistral) — 14.8 (2024-04-17) Llama 3 Instruct 70B (Meta) — 19.8 (2024-04-18) Llama 3 Instruct 8B (Meta) — 9.6 (2024-04-18) Phi-3 Mini Instruct 3.8B (Microsoft) — 11.6 (2024-04-23) GPT-4o (May '24) (OpenAI) — 33.4 (2024-05-13) Gemini 1.5 Flash (May '24) (Google) — 19.6 (2024-05-14) Gemini 1.5 Pro (May '24) (Google) — 24.4 (2024-05-15) Qwen2 Instruct 72B (Alibaba) — 15.9 (2024-06-07) DeepSeek Coder V2 Lite Instruct (DeepSeek) — 15.8 (2024-06-17) GPT-4o mini (OpenAI) — 23.4 (2024-07-18) Llama 3.1 Instruct 405B (Meta) — 30.5 (2024-07-23) Llama 3.1 Instruct 70B (Meta) — 23.2 (2024-07-23) Llama 3.1 Instruct 8B (Meta) — 11.6 (2024-07-23) Mistral Large 2 (Jul '24) (Mistral) — 26.7 (2024-07-24) GPT-4o (Aug '24) (OpenAI) — 31.7 (2024-08-06) Grok Beta (SpaceXAI) — 24.1 (2024-08-13) Hermes 3 - Llama-3.1 70B (Nous Research) — 18.8 (2024-08-15) Jamba 1.5 Large (AI21 Labs) — 14.3 (2024-08-22) Jamba 1.5 Mini (AI21 Labs) — 6.2 (2024-08-22) o1-mini (OpenAI) — 57.6 (2024-09-12) Mistral Small (Sep '24) (Mistral) — 14.1 (2024-09-17) Qwen2.5 Instruct 72B (Alibaba) — 27.6 (2024-09-19) Qwen2.5 Instruct 32B (Alibaba) — 24.8 (2024-09-19) Qwen2.5 Coder Instruct 7B (Alibaba) — 12.6 (2024-09-19) Gemini 1.5 Pro (Sep '24) (Google) — 31.6 (2024-09-24) Gemini 1.5 Flash (Sep '24) (Google) — 27.3 (2024-09-24) Llama 3.2 Instruct 90B (Vision) (Meta) — 21.4 (2024-09-25) Llama 3.2 Instruct 11B (Vision) (Meta) — 11.0 (2024-09-25) Llama 3.2 Instruct 3B (Meta) — 8.3 (2024-09-25) Molmo 7B-D (Allen Institute for AI) — 3.9 (2024-09-25) Llama 3.2 Instruct 1B (Meta) — 1.9 (2024-09-25) LFM 40B (Liquid AI) — 9.6 (2024-09-30) Gemini 1.5 Flash-8B (Google) — 21.7 (2024-10-03) Llama 3.1 Nemotron Instruct 70B (NVIDIA) — 16.9 (2024-10-15) Claude 3.5 Sonnet (Oct '24) (Anthropic) — 38.1 (2024-10-22) Claude 3.5 Haiku (Anthropic) — 31.4 (2024-10-22) Qwen2.5 Coder Instruct 32B (Alibaba) — 29.5 (2024-11-11) Mistral Large 2 (Nov '24) (Mistral) — 29.3 (2024-11-18) Pixtral Large (Mistral) — 26.1 (2024-11-18) Qwen2.5 Turbo (Alibaba) — 16.3 (2024-11-18) GPT-4o (Nov '24) (OpenAI) — 30.9 (2024-11-20) OLMo 2 7B (Allen Institute for AI) — 4.1 (2024-11-26) QwQ 32B-Preview (Alibaba) — 33.7 (2024-11-27) Nova Pro (Amazon) — 23.3 (2024-12-03) Nova Lite (Amazon) — 16.7 (2024-12-03) Nova Micro (Amazon) — 14.0 (2024-12-03) o1 (OpenAI) — 67.9 (2024-12-05) Llama 3.3 Instruct 70B (Meta) — 28.8 (2024-12-06) Gemini 2.0 Flash (experimental) (Google) — 21.0 (2024-12-11) Grok 2 (Dec '24) (SpaceXAI) — 26.7 (2024-12-12) Phi-4 (Microsoft) — 23.1 (2024-12-12) DeepSeek V3 (Dec '24) (DeepSeek) — 35.9 (2024-12-26) DeepSeek R1 (Jan '25) (DeepSeek) — 61.7 (2025-01-20) DeepSeek R1 Distill Qwen 14B (DeepSeek) — 37.6 (2025-01-20) DeepSeek R1 Distill Qwen 32B (DeepSeek) — 27.0 (2025-01-20) DeepSeek R1 Distill Llama 70B (DeepSeek) — 26.6 (2025-01-20) DeepSeek R1 Distill Llama 8B (DeepSeek) — 23.3 (2025-01-20) DeepSeek R1 Distill Qwen 1.5B (DeepSeek) — 7.0 (2025-01-20) Gemini 2.0 Flash Thinking Experimental (Jan '25) (Google) — 32.1 (2025-01-21) Sonar (Perplexity) — 29.5 (2025-01-21) Sonar Pro (Perplexity) — 27.5 (2025-01-21) Qwen2.5 Max (Alibaba) — 35.9 (2025-01-28) Llama 3.1 Tulu3 405B (Allen Institute for AI) — 29.1 (2025-01-30) Mistral Small 3 (Mistral) — 25.2 (2025-01-30) o3-mini (high) (OpenAI) — 73.4 (2025-01-31) o3-mini (OpenAI) — 71.7 (2025-01-31) Gemini 2.0 Pro Experimental (Feb '25) (Google) — 34.7 (2025-02-05) Gemini 2.0 Flash (Feb '25) (Google) — 33.4 (2025-02-05) Gemini 2.0 Flash-Lite (Preview) (Google) — 17.9 (2025-02-05) DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) (Nous Research) — 8.5 (2025-02-13) Grok 3 mini Reasoning (high) (SpaceXAI) — 69.6 (2025-02-19) Grok 3 (SpaceXAI) — 42.5 (2025-02-19) Claude 3.7 Sonnet (Reasoning) (Anthropic) — 47.3 (2025-02-24) Claude 3.7 Sonnet (Non-reasoning) (Anthropic) — 39.4 (2025-02-24) Gemini 2.0 Flash-Lite (Feb '25) (Google) — 18.5 (2025-02-25) Phi-4 Multimodal Instruct (Microsoft) — 13.1 (2025-02-26) QwQ 32B (Alibaba) — 63.1 (2025-03-05) Jamba 1.6 Large (AI21 Labs) — 17.2 (2025-03-06) Jamba 1.6 Mini (AI21 Labs) — 7.1 (2025-03-06) Reka Flash 3 (Reka AI) — 43.5 (2025-03-10) Gemma 3 27B Instruct (Google) — 13.7 (2025-03-12) Gemma 3 12B Instruct (Google) — 13.7 (2025-03-12) Gemma 3 4B Instruct (Google) — 11.2 (2025-03-12) Command A (Cohere) — 28.7 (2025-03-13) DeepHermes 3 - Mistral 24B Preview (Non-reasoning) (Nous Research) — 19.5 (2025-03-13) OLMo 2 32B (Allen Institute for AI) — 6.8 (2025-03-13) Gemma 3 1B Instruct (Google) — 1.7 (2025-03-13) Mistral Small 3.1 (Mistral) — 21.2 (2025-03-17) Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) (NVIDIA) — 28.0 (2025-03-18) Llama 3.3 Nemotron Super 49B v1 (Reasoning) (NVIDIA) — 27.7 (2025-03-18) Gemini 2.5 Pro Preview (Mar' 25) (Google) — 77.8 (2025-03-25) DeepSeek V3 0324 (DeepSeek) — 40.5 (2025-03-25) GPT-4o (March 2025, chatgpt-4o-latest) (OpenAI) — 42.5 (2025-03-27) Llama 4 Maverick (Meta) — 39.7 (2025-04-05) Llama 4 Scout (Meta) — 29.9 (2025-04-05) Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) (NVIDIA) — 64.1 (2025-04-07) GPT-4.1 mini (OpenAI) — 48.3 (2025-04-14) GPT-4.1 (OpenAI) — 45.7 (2025-04-14) GPT-4.1 nano (OpenAI) — 32.6 (2025-04-14) o4-mini (high) (OpenAI) — 85.9 (2025-04-16) o3 (OpenAI) — 80.8 (2025-04-16) Granite 3.3 8B (Non-reasoning) (IBM) — 12.7 (2025-04-16) Gemini 2.5 Flash Preview (Reasoning) (Google) — 50.5 (2025-04-17) Gemini 2.5 Flash Preview (Non-reasoning) (Google) — 40.6 (2025-04-17) Qwen3 235B A22B (Reasoning) (Alibaba) — 62.2 (2025-04-28) Qwen3 32B (Reasoning) (Alibaba) — 54.6 (2025-04-28) Qwen3 14B (Reasoning) (Alibaba) — 52.3 (2025-04-28) Qwen3 30B A3B (Reasoning) (Alibaba) — 50.6 (2025-04-28) Qwen3 4B (Reasoning) (Alibaba) — 46.5 (2025-04-28) Qwen3 8B (Reasoning) (Alibaba) — 40.6 (2025-04-28) Qwen3 235B A22B (Non-reasoning) (Alibaba) — 34.3 (2025-04-28) Qwen3 30B A3B (Non-reasoning) (Alibaba) — 32.2 (2025-04-28) Qwen3 1.7B (Reasoning) (Alibaba) — 30.8 (2025-04-28) Qwen3 32B (Non-reasoning) (Alibaba) — 28.8 (2025-04-28) Qwen3 14B (Non-reasoning) (Alibaba) — 28.0 (2025-04-28) Qwen3 4B (Non-reasoning) (Alibaba) — 23.3 (2025-04-28) Qwen3 8B (Non-reasoning) (Alibaba) — 20.2 (2025-04-28) Qwen3 1.7B (Non-reasoning) (Alibaba) — 12.6 (2025-04-28) Qwen3 0.6B (Reasoning) (Alibaba) — 12.1 (2025-04-28) Qwen3 0.6B (Non-reasoning) (Alibaba) — 7.3 (2025-04-28) Nova Premier (Amazon) — 31.7 (2025-04-30) Gemini 2.5 Pro Preview (May' 25) (Google) — 77.0 (2025-05-06) Mistral Medium 3 (Mistral) — 40.0 (2025-05-07) Gemini 2.5 Flash (Reasoning) (Google) — 69.5 (2025-05-20) Gemini 2.5 Flash (Non-reasoning) (Google) — 49.5 (2025-05-20) Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) (NVIDIA) — 49.3 (2025-05-20) Solar Pro 2 (Preview) (Reasoning) (Upstage) — 46.2 (2025-05-20) Solar Pro 2 (Preview) (Non-reasoning) (Upstage) — 38.5 (2025-05-20) Gemma 3n E4B Instruct Preview (May '25) (Google) — 13.8 (2025-05-20) Devstral Small (May '25) (Mistral) — 25.8 (2025-05-21) Claude 4 Sonnet (Reasoning) (Anthropic) — 65.5 (2025-05-22) Claude 4 Opus (Reasoning) (Anthropic) — 63.6 (2025-05-22) Claude 4 Opus (Non-reasoning) (Anthropic) — 54.2 (2025-05-22) Claude 4 Sonnet (Non-reasoning) (Anthropic) — 44.9 (2025-05-22) Sarvam M (Reasoning) (Sarvam) — 29.5 (2025-05-23) DeepSeek R1 0528 (May '25) (DeepSeek) — 77.0 (2025-05-28) DeepSeek R1 0528 Qwen3 8B (DeepSeek) — 51.3 (2025-05-29) Gemini 2.5 Pro (Google) — 80.1 (2025-06-05) Magistral Medium 1 (Mistral) — 52.7 (2025-06-10) Magistral Small 1 (Mistral) — 51.4 (2025-06-10) MiniMax M1 80k (MiniMax) — 71.1 (2025-06-17) MiniMax M1 40k (MiniMax) — 65.7 (2025-06-17) Gemini 2.5 Flash-Lite (Reasoning) (Google) — 59.3 (2025-06-17) Gemini 2.5 Flash-Lite (Non-reasoning) (Google) — 40.0 (2025-06-17) Mistral Small 3.2 (Mistral) — 27.5 (2025-06-20) Gemma 3n E4B Instruct (Google) — 14.6 (2025-06-26) Gemma 3n E2B Instruct (Google) — 9.5 (2025-06-26) ERNIE 4.5 300B A47B (Baidu) — 46.7 (2025-06-30) Jamba 1.7 Large (AI21 Labs) — 18.1 (2025-07-07) Jamba 1.7 Mini (AI21 Labs) — 6.1 (2025-07-07) Solar Pro 2 (Reasoning) (Upstage) — 61.6 (2025-07-09) Solar Pro 2 (Non-reasoning) (Upstage) — 42.4 (2025-07-09) Grok 4 (SpaceXAI) — 81.9 (2025-07-10) Devstral Medium (Mistral) — 33.7 (2025-07-10) Devstral Small (Jul '25) (Mistral) — 25.4 (2025-07-10) LFM2 1.2B (Liquid AI) — 2.0 (2025-07-10) Kimi K2 (Kimi) — 55.6 (2025-07-11) EXAONE 4.0 32B (Reasoning) (LG AI Research) — 74.7 (2025-07-15) Exaone 4.0 1.2B (Reasoning) (LG AI Research) — 51.6 (2025-07-15) EXAONE 4.0 32B (Non-reasoning) (LG AI Research) — 47.2 (2025-07-15) Exaone 4.0 1.2B (Non-reasoning) (LG AI Research) — 29.3 (2025-07-15) Qwen3 235B A22B 2507 Instruct (Alibaba) — 52.4 (2025-07-21) Qwen3 Coder 480B A35B Instruct (Alibaba) — 58.5 (2025-07-22) Qwen3 235B A22B 2507 (Reasoning) (Alibaba) — 78.8 (2025-07-25) Llama Nemotron Super 49B v1.5 (Reasoning) (NVIDIA) — 73.7 (2025-07-25) Llama Nemotron Super 49B v1.5 (Non-reasoning) (NVIDIA) — 29.0 (2025-07-25) GLM-4.5 (Reasoning) (Z AI) — 73.8 (2025-07-28) GLM-4.5-Air (Z AI) — 68.4 (2025-07-28) Qwen3 30B A3B 2507 Instruct (Alibaba) — 51.5 (2025-07-29) Qwen3 30B A3B 2507 (Reasoning) (Alibaba) — 70.7 (2025-07-30) Qwen3 Coder 30B A3B Instruct (Alibaba) — 40.3 (2025-07-31) gpt-oss-120b (high) (OpenAI) — 87.8 (2025-08-05) gpt-oss-20b (high) (OpenAI) — 77.7 (2025-08-05) gpt-oss-120b (low) (OpenAI) — 70.7 (2025-08-05) Claude 4.1 Opus (Reasoning) (Anthropic) — 65.4 (2025-08-05) gpt-oss-20b (low) (OpenAI) — 65.2 (2025-08-05) Qwen3 4B 2507 (Reasoning) (Alibaba) — 64.1 (2025-08-06) Qwen3 4B 2507 Instruct (Alibaba) — 37.7 (2025-08-06) GPT-5 (high) (OpenAI) — 84.6 (2025-08-07) GPT-5 mini (high) (OpenAI) — 83.8 (2025-08-07) GPT-5 nano (high) (OpenAI) — 78.9 (2025-08-07) GPT-5 (low) (OpenAI) — 76.3 (2025-08-07) GPT-5 nano (medium) (OpenAI) — 76.3 (2025-08-07) GPT-5 (medium) (OpenAI) — 70.3 (2025-08-07) GPT-5 mini (medium) (OpenAI) — 69.2 (2025-08-07) GPT-5 (minimal) (OpenAI) — 55.8 (2025-08-07) GPT-5 mini (minimal) (OpenAI) — 54.5 (2025-08-07) GPT-5 (ChatGPT) (OpenAI) — 54.3 (2025-08-07) GPT-5 nano (minimal) (OpenAI) — 47.0 (2025-08-07) GLM-4.5V (Reasoning) (Z AI) — 60.4 (2025-08-11) GLM-4.5V (Non-reasoning) (Z AI) — 35.2 (2025-08-11) Mistral Medium 3.1 (Mistral) — 40.6 (2025-08-12) Gemma 3 270M (Google) — 0.3 (2025-08-14) NVIDIA Nemotron Nano 9B V2 (Reasoning) (NVIDIA) — 72.4 (2025-08-18) NVIDIA Nemotron Nano 9B V2 (Non-reasoning) (NVIDIA) — 70.1 (2025-08-18) Seed-OSS-36B-Instruct (ByteDance Seed) — 76.5 (2025-08-20) DeepSeek V3.1 (Reasoning) (DeepSeek) — 78.4 (2025-08-21) DeepSeek V3.1 (Non-reasoning) (DeepSeek) — 57.7 (2025-08-21) Hermes 4 - Llama-3.1 405B (Reasoning) (Nous Research) — 68.6 (2025-08-27) Hermes 4 - Llama-3.1 70B (Reasoning) (Nous Research) — 65.3 (2025-08-27) Hermes 4 - Llama-3.1 405B (Non-reasoning) (Nous Research) — 54.6 (2025-08-27) Hermes 4 - Llama-3.1 70B (Non-reasoning) (Nous Research) — 26.9 (2025-08-27) Grok Code Fast 1 (SpaceXAI) — 65.7 (2025-08-28) Dream-Coder — 21.4 (2025-09-01) Qwen3 Max (Preview) (Alibaba) — 65.1 (2025-09-05) Kimi K2 0905 (Kimi) — 61.0 (2025-09-05) Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning) (Google) — 68.8 (2025-09-08) Ling-mini-2.0 (InclusionAI) — 42.9 (2025-09-09) Qwen3 Next 80B A3B (Reasoning) (Alibaba) — 78.4 (2025-09-11) Qwen3 Next 80B A3B Instruct (Alibaba) — 68.4 (2025-09-11) Magistral Small 1.2 (Mistral) — 72.3 (2025-09-17) Ling-flash-2.0 (InclusionAI) — 58.9 (2025-09-17) Magistral Medium 1.2 (Mistral) — 75.0 (2025-09-18) Grok 4 Fast (Reasoning) (SpaceXAI) — 83.2 (2025-09-19) Ring-flash-2.0 (InclusionAI) — 62.8 (2025-09-19) Grok 4 Fast (Non-reasoning) (SpaceXAI) — 40.1 (2025-09-19) DeepSeek V3.1 Terminus (Reasoning) (DeepSeek) — 79.8 (2025-09-22) Qwen3 Omni 30B A3B (Reasoning) (Alibaba) — 67.9 (2025-09-22) DeepSeek V3.1 Terminus (Non-reasoning) (DeepSeek) — 52.9 (2025-09-22) Qwen3 Omni 30B A3B Instruct (Alibaba) — 42.2 (2025-09-22) Granite 4.0 H Small (IBM) — 25.1 (2025-09-22) Granite 4.0 Micro (IBM) — 18.0 (2025-09-22) GPT-5 Codex (high) (OpenAI) — 84.0 (2025-09-23) Qwen3 Max (Alibaba) — 76.7 (2025-09-23) Qwen3 VL 235B A22B (Reasoning) (Alibaba) — 64.6 (2025-09-23) Qwen3 VL 235B A22B Instruct (Alibaba) — 59.4 (2025-09-23) LFM2 2.6B (Liquid AI) — 8.1 (2025-09-23) Gemini 2.5 Flash Preview (Sep '25) (Reasoning) (Google) — 71.3 (2025-09-25) Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning) (Google) — 64.1 (2025-09-25) Gemini 2.5 Flash Preview (Sep '25) (Non-reasoning) (Google) — 62.5 (2025-09-25) DeepSeek V3.2 Exp (Reasoning) (DeepSeek) — 78.9 (2025-09-29) Claude 4.5 Sonnet (Reasoning) (Anthropic) — 71.4 (2025-09-29) Claude 4.5 Sonnet (Non-reasoning) (Anthropic) — 59.0 (2025-09-29) DeepSeek V3.2 Exp (Non-reasoning) (DeepSeek) — 55.4 (2025-09-29) Apriel-v1.5-15B-Thinker (ServiceNow) — 72.8 (2025-09-30) GLM-4.6 (Reasoning) (Z AI) — 69.5 (2025-09-30) CWM — 68.6 (2025-09-30) GLM-4.6 (Non-reasoning) (Z AI) — 56.1 (2025-09-30) Qwen3 VL 30B A3B (Reasoning) (Alibaba) — 69.7 (2025-10-03) Qwen3 VL 30B A3B Instruct (Alibaba) — 47.6 (2025-10-03) LFM2 8B A1B (Liquid AI) — 15.1 (2025-10-07) Ling-1T (InclusionAI) — 67.7 (2025-10-08) Jamba Reasoning 3B (AI21 Labs) — 21.0 (2025-10-08) Ring-1T (InclusionAI) — 64.3 (2025-10-13) Qwen3 VL 8B (Reasoning) (Alibaba) — 35.3 (2025-10-14) Qwen3 VL 8B Instruct (Alibaba) — 33.2 (2025-10-14) Qwen3 VL 4B (Reasoning) (Alibaba) — 32.0 (2025-10-14) Qwen3 VL 4B Instruct (Alibaba) — 29.0 (2025-10-14) Claude 4.5 Haiku (Reasoning) (Anthropic) — 61.5 (2025-10-15) Claude 4.5 Haiku (Non-reasoning) (Anthropic) — 51.1 (2025-10-15) Qwen3 VL 32B (Reasoning) (Alibaba) — 73.8 (2025-10-21) Qwen3 VL 32B Instruct (Alibaba) — 51.4 (2025-10-21) MiniMax-M2 (MiniMax) — 82.6 (2025-10-26) NVIDIA Nemotron Nano 12B v2 VL (Reasoning) (NVIDIA) — 69.4 (2025-10-28) NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning) (NVIDIA) — 34.5 (2025-10-28) Granite 4.0 H 1B (IBM) — 11.5 (2025-10-28) Granite 4.0 1B (IBM) — 4.7 (2025-10-28) Granite 4.0 350M (IBM) — 2.4 (2025-10-28) Granite 4.0 H 350M (IBM) — 1.9 (2025-10-28) Nova 2.0 Lite (high) (Amazon) — 71.1 (2025-10-29) Nova 2.0 Lite (medium) (Amazon) — 66.3 (2025-10-29) Nova 2.0 Lite (low) (Amazon) — 46.9 (2025-10-29) Nova 2.0 Lite (Non-reasoning) (Amazon) — 34.6 (2025-10-29) Kimi Linear 48B A3B Instruct (Kimi) — 37.8 (2025-10-30) Qwen3 Max Thinking (Preview) (Alibaba) — 53.5 (2025-11-03) Kimi K2 Thinking (Kimi) — 85.3 (2025-11-06) Doubao Seed Code (ByteDance Seed) — 76.6 (2025-11-11) KAT-Coder-Pro V1 (KwaiKAT) — 74.7 (2025-11-11) GPT-5.1 (high) (OpenAI) — 86.8 (2025-11-13) GPT-5.1 Codex (high) (OpenAI) — 84.9 (2025-11-13) GPT-5.1 Codex mini (high) (OpenAI) — 83.6 (2025-11-13) ERNIE 5.0 Thinking Preview (Baidu) — 81.2 (2025-11-13) GPT-5.1 (Non-reasoning) (OpenAI) — 49.4 (2025-11-13) Gemini 3 Pro Preview (high) (Google) — 91.7 (2025-11-18) Gemini 3 Pro Preview (low) (Google) — 85.7 (2025-11-18) Cogito v2.1 (Reasoning) (Deep Cogito) — 68.8 (2025-11-18) Grok 4.1 Fast (Reasoning) (SpaceXAI) — 82.2 (2025-11-19) Grok 4.1 Fast (Non-reasoning) (SpaceXAI) — 39.9 (2025-11-19) Olmo 3 32B Think (Allen Institute for AI) — 67.2 (2025-11-20) Olmo 3 7B Think (Allen Institute for AI) — 61.7 (2025-11-20) Olmo 3 7B Instruct (Allen Institute for AI) — 26.6 (2025-11-20) Claude Opus 4.5 (Reasoning) (Anthropic) — 87.1 (2025-11-24) Claude Opus 4.5 (Non-reasoning) (Anthropic) — 73.8 (2025-11-24) Apriel-v1.6-15B-Thinker (ServiceNow) — 80.7 (2025-11-25) Nova 2.0 Omni (medium) (Amazon) — 66.0 (2025-11-26) Nova 2.0 Omni (low) (Amazon) — 59.2 (2025-11-26) Nova 2.0 Omni (Non-reasoning) (Amazon) — 30.5 (2025-11-26) INTELLECT-3 (Prime Intellect) — 77.7 (2025-11-27) Nova 2.0 Pro Preview (medium) (Amazon) — 73.0 (2025-11-27) Nova 2.0 Pro Preview (low) (Amazon) — 63.8 (2025-11-27) Nova 2.0 Pro Preview (Non-reasoning) (Amazon) — 47.3 (2025-11-27) DeepSeek V3.2 Speciale (DeepSeek) — 89.6 (2025-12-01) DeepSeek V3.2 (Reasoning) (DeepSeek) — 86.2 (2025-12-01) DeepSeek V3.2 (Non-reasoning) (DeepSeek) — 59.3 (2025-12-01) Mistral Large 3 (Mistral) — 46.5 (2025-12-02) Ministral 3 14B (Mistral) — 35.1 (2025-12-02) Ministral 3 8B (Mistral) — 30.3 (2025-12-02) Ministral 3 3B (Mistral) — 24.7 (2025-12-02) ThinkMerge — 8.28 (2025-12-02) Motif-2-12.7B-Reasoning (Motif Technologies) — 65.1 (2025-12-04) K2-V2 (high) (MBZUAI Institute of Foundation Models) — 69.4 (2025-12-05) K2-V2 (medium) (MBZUAI Institute of Foundation Models) — 54.1 (2025-12-05) K2-V2 (low) (MBZUAI Institute of Foundation Models) — 39.3 (2025-12-05) GLM-4.6V (Non-reasoning) (Z AI) — 41.1 (2025-12-08) GLM-4.6V (Reasoning) (Z AI) — 16.0 (2025-12-08) Devstral 2 (Mistral) — 44.8 (2025-12-09) Devstral Small 2 (Mistral) — 34.8 (2025-12-09) GPT-5.2 (medium) (OpenAI) — 89.4 (2025-12-11) GPT-5.2 (xhigh) (OpenAI) — 88.9 (2025-12-11) GPT-5.2 (Non-reasoning) (OpenAI) — 66.9 (2025-12-11) Mi:dm K 2.5 Pro (Korea Telecom) — 65.6 (2025-12-11) Mi:dm K 2.5 Pro Preview (Korea Telecom) — 57.6 (2025-12-11) Olmo 3.1 32B Think (Allen Institute for AI) — 69.5 (2025-12-12) NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) (NVIDIA) — 74.1 (2025-12-15) NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning) (NVIDIA) — 36.0 (2025-12-15) MiMo-V2-Flash (Reasoning) (Xiaomi) — 86.8 (2025-12-16) MiMo-V2-Flash (Non-reasoning) (Xiaomi) — 40.2 (2025-12-16) Gemini 3 Flash Preview (Reasoning) (Google) — 90.8 (2025-12-17) DreamPRM-Code — 80.9 (2025-12-17) Gemini 3 Flash Preview (Non-reasoning) (Google) — 79.7 (2025-12-17) GLM-4.7 (Reasoning) (Z AI) — 89.4 (2025-12-22) GLM-4.7 (Non-reasoning) (Z AI) — 56.2 (2025-12-22) MiniMax-M2.1 (MiniMax) — 81.0 (2025-12-23) HyperCLOVA X SEED Think (32B) (Naver) — 62.9 (2025-12-26) K-EXAONE (Reasoning) (LG AI Research) — 76.8 (2025-12-31) Falcon-H1R-7B (TII UAE) — 72.4 (2026-01-04) Bridging Online and Offline RL: Contextu — 6.2 (2026-02-03) Embarrassingly Simple Self-Distillation — 55.3 (2026-04-01) DuST — 6.2 (2026-05-11) Claude Instant (Anthropic) — 10.9 (2023-03-14) Claude 2.0 (Anthropic) — 17.1 (2023-07-11) GPT-4 Turbo (OpenAI) — 29.1 (2023-11-06) GPT-4o (May '24) (OpenAI) — 33.4 (2024-05-13) o1-mini (OpenAI) — 57.6 (2024-09-12) o1 (OpenAI) — 67.9 (2024-12-05) o3-mini (high) (OpenAI) — 73.4 (2025-01-31) Gemini 2.5 Pro Preview (Mar' 25) (Google) — 77.8 (2025-03-25) o4-mini (high) (OpenAI) — 85.9 (2025-04-16) gpt-oss-120b (high) (OpenAI) — 87.8 (2025-08-05) Gemini 3 Pro Preview (high) (Google) — 91.7 (2025-11-18)
RankModel Pass@1 PaperCodeYear
1 Gemini 3 Pro Preview (high) (Google) 외부 91.7 artificialanalysis.ai 2025
2 Gemini 3 Flash Preview (Reasoning) (Google) 외부 90.8 artificialanalysis.ai 2025
3 DeepSeek V3.2 Speciale (DeepSeek) 외부 89.6 artificialanalysis.ai 2025
4 GPT-5.2 (medium) (OpenAI) 외부 89.4 artificialanalysis.ai 2025
4 GLM-4.7 (Reasoning) (Z AI) 외부 89.4 artificialanalysis.ai 2025
6 GPT-5.2 (xhigh) (OpenAI) 외부 88.9 artificialanalysis.ai 2025
7 gpt-oss-120b (high) (OpenAI) 외부 87.8 artificialanalysis.ai 2025
8 Claude Opus 4.5 (Reasoning) (Anthropic) 외부 87.1 artificialanalysis.ai 2025
9 GPT-5.1 (high) (OpenAI) 외부 86.8 artificialanalysis.ai 2025
9 MiMo-V2-Flash (Reasoning) (Xiaomi) 외부 86.8 artificialanalysis.ai 2025
11 DeepSeek V3.2 (Reasoning) (DeepSeek) 외부 86.2 artificialanalysis.ai 2025
12 o4-mini (high) (OpenAI) 외부 85.9 artificialanalysis.ai 2025
13 Gemini 3 Pro Preview (low) (Google) 외부 85.7 artificialanalysis.ai 2025
14 Kimi K2 Thinking (Kimi) 외부 85.3 artificialanalysis.ai 2025
15 GPT-5.1 Codex (high) (OpenAI) 외부 84.9 artificialanalysis.ai 2025
16 GPT-5 (high) (OpenAI) 외부 84.6 artificialanalysis.ai 2025
17 GPT-5 Codex (high) (OpenAI) 외부 84 artificialanalysis.ai 2025
18 GPT-5 mini (high) (OpenAI) 외부 83.8 artificialanalysis.ai 2025
19 GPT-5.1 Codex mini (high) (OpenAI) 외부 83.6 artificialanalysis.ai 2025
20 Grok 4 Fast (Reasoning) (SpaceXAI) 외부 83.2 artificialanalysis.ai 2025
1–20 / 350 다음 → 페이지당 10 20 50 100