Code Generation 벤치마크
Code Generation on DSEval-LeetCode
| Rank | Model | Pass Rate | w/o Intact | w/o PE | Paper | Code | Year |
|---|---|---|---|---|---|---|---|
| 1 | Jupyter-AI | 57.5 | 57.5 | 70.0 | |||
| 2 | Chapyter | 45.0 | 45.0 | 60.0 | |||
| 2 | Code Interpreter API | 45.0 | 45.0 | 55.0 | |||
| 4 | CoML | 42.5 | 42.5 | 62.5 | MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks | microsoft/CoML | 2023 |
| 5 | ChatDev | 32.5 | 32.5 | 50.0 |