paper-with-me

Code Generation 벤치마크

Code Generation on HumanEval

12개 결과 · ⬇ CSV · JSON

Pass@1

90.2 92.65 95.1 97.55 100 2023-10 2026-09 L2MAC (GPT-4) — 90.2 (2023-10-02) LLaMA 3 — 99.4 (2024-02-25) Mistral 7B — 93.9 (2024-05-18) DeepSeek-R1 (MGDebugger) — 100.0 (2024-10-02) Phi-2 — 98.2 (2024-11-21) QualityFlow (Sonnet-3.5) — 98.8 (2025-01-20) EG-CFG (DeepSeek-V3-0324) — 96.95 (2025-06-12) Multi-task Code LLMs — 92.7 (2026-01-28) DebateCoder — 95.0 (2026-01-29) How Many Tries Does It Take? Iterative S — 96.3 (2026-04-12) SolidCoder — 95.7 (2026-04-20) L2MAC (GPT-4) — 90.2 (2023-10-02) LLaMA 3 — 99.4 (2024-02-25) DeepSeek-R1 (MGDebugger) — 100.0 (2024-10-02)
RankModel Pass@1 PaperCodeYear
1 DeepSeek-R1 (MGDebugger) 100 From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging YerbaPage/MGDebugger 2024
2 LLaMA 3 99.4 Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step floridsleeves/llmdebugger 2024
3 QualityFlow (Sonnet-3.5) 98.8 QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks 2025
4 Phi-2 98.2 Planning-Driven Programming: A Large Language Model Programming Workflow you68681/lpw 2024
5 EG-CFG (DeepSeek-V3-0324) 96.95 Execution Guided Line-by-Line Code Generation boazlavon/eg_cfg 2025
6 How Many Tries Does It Take? Iterative S 자동 추출 96.3 How Many Tries Does It Take? Iterative Self-Repair in LLM Code Generation Across Model Scales and Benchmarks 2026
7 SolidCoder 자동 추출 95.7 SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution 2026
8 DebateCoder 자동 추출 95 Adaptive Confidence Gating in Multi-Agent Collaboration for Efficient and Optimized Code Generation 2026
9 Mistral 7B 93.9 MapCoder: Multi-Agent Code Generation for Competitive Problem Solving md-ashraful-pramanik/mapcoder · Luoji-zju/Agents4PLC_release 2024
10 Multi-task Code LLMs 자동 추출 92.7 Multi-task Code LLMs: Data Mix or Model Merge? 2026
11 Claude Sonnet 3.5 90.85
12 L2MAC (GPT-4) 90.2 L2MAC: Large Language Model Automatic Computer for Extensive Code Generation samholt/l2mac · vanderschaarlab/l2mac 2023
1–12 / 12 페이지당 10 20 50 100