Code Generation 벤치마크
Code Generation on SciCode
Accuracy
- 2024-12-26 — DeepSeek V3 (Dec '24) (DeepSeek): Accuracy 35.8
- 2025-01-20 — DeepSeek R1 (Jan '25) (DeepSeek): Accuracy 38.3
- 2025-01-31 — o3-mini (high) (OpenAI): Accuracy 42.8
- 2025-06-05 — Gemini 2.5 Pro (Google): Accuracy 46.3
- 2026-02-17 — Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) (Anthropic): Accuracy 50.1
- 2026-02-19 — Gemini 3.1 Pro Preview (Google): Accuracy 58.7
- 2026-06-09 — Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (Anthropic): Accuracy 61.0
- 2026-09-01 — Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (Anthropic): Accuracy 63.1
| Rank | Model | Accuracy | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (Anthropic) 외부 | 63.1 | artificialanalysis.ai | 2026 | |
| 2 | Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (Anthropic) 외부 | 61 | artificialanalysis.ai | 2026 | |
| 3 | Claude Fable 5.1 (Adaptive Reasoning, Xhigh Effort, Default Fallback) (Anthropic) 외부 | 60.9 | artificialanalysis.ai | 2026 | |
| 4 | Gemini 3.7 Flash (medium) (Google) 외부 | 59.8 | artificialanalysis.ai | 2026 | |
| 5 | Muse Spark 1.3 (xhigh) (Meta) 외부 | 59.7 | artificialanalysis.ai | 2026 | |
| 6 | Kimi K3 (max) (Kimi) 외부 | 59.5 | artificialanalysis.ai | 2026 | |
| 7 | GLM-5.3 (max) (Z AI) 외부 | 59 | artificialanalysis.ai | 2026 | |
| 8 | Muse Spark 1.3 (max) (Meta) 외부 | 58.8 | artificialanalysis.ai | 2026 | |
| 8 | Muse Spark 1.1 (xhigh) (Meta) 외부 | 58.8 | artificialanalysis.ai | 2026 | |
| 10 | Claude Fable 5.1 (Adaptive Reasoning, High Effort, Default Fallback) (Anthropic) 외부 | 58.7 | artificialanalysis.ai | 2026 | |
| 10 | Gemini 3.1 Pro Preview (Google) 외부 | 58.7 | artificialanalysis.ai | 2026 | |
| 12 | GPT-5.6 Sol (high) (OpenAI) 외부 | 57.8 | artificialanalysis.ai | 2026 | |
| 13 | Muse Spark 1.2 (xhigh) (Meta) 외부 | 57.4 | artificialanalysis.ai | 2026 | |
| 13 | GPT-5.6 Sol (medium) (OpenAI) 외부 | 57.4 | artificialanalysis.ai | 2026 | |
| 15 | Gemini 3.7 Flash (high) (Google) 외부 | 57.2 | artificialanalysis.ai | 2026 | |
| 16 | GPT-5.6 Sol (max) (OpenAI) 외부 | 57.1 | artificialanalysis.ai | 2026 | |
| 16 | GPT-5.6 Sol (xhigh) (OpenAI) 외부 | 57.1 | artificialanalysis.ai | 2026 | |
| 18 | Claude Fable 5.1 (Adaptive Reasoning, Low Effort, Default Fallback) (Anthropic) 외부 | 56.7 | artificialanalysis.ai | 2026 | |
| 19 | Gemini 3.8 Flash (high) (Google) 외부 | 56.6 | artificialanalysis.ai | 2026 | |
| 20 | GPT-6 Astra (max) (OpenAI) 외부 | 56.5 | artificialanalysis.ai | 2026 |