Code Generation 벤치마크
Code Generation on TACO-BAAI
easy pass@1
- 2023-12-22 — GPT-4: easy pass@1 31.5
| Rank | Model | easy pass@1 | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | GPT-4 | 31.50% | TACO: Topics in Algorithmic COde generation dataset | flagopen/taco | 2023 |
| 2 | Starcoder-15.5B | 11.6% | TACO: Topics in Algorithmic COde generation dataset | flagopen/taco | 2023 |
| 3 | CodeLlama-7B-Python | 9.32% | TACO: Topics in Algorithmic COde generation dataset | flagopen/taco | 2023 |