Code Generation 벤치마크
Code Generation on BigCodeBench-Complete
| Rank | Model | Pass@1 | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | GPT-4o-2024-05-13 | 61.1 | BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions | mlfoundations/Evalchemy · bigcode-project/bigcodebench · bigcode-project/bigcodebench-annotation · +1 | 2024 |
| 2 | DeepSeek-Coder-V2-Instruct | 59.7 | BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions | mlfoundations/Evalchemy · bigcode-project/bigcodebench · bigcode-project/bigcodebench-annotation · +1 | 2024 |