Code Generation 벤치마크
Code Generation on WebApp1k-Duo-React
pass@1
- 2024-09-19 — claude-3-5-sonnet: pass@1 0.679
| Rank | Model | pass@1 | Paper | Code | Year |
|---|---|---|---|---|---|
| 1 | claude-3-5-sonnet | 0.679 | A Case Study of Web App Coding with OpenAI Reasoning Models | onekq/webapp1k | 2024 |
| 2 | o1-mini | 0.667 | A Case Study of Web App Coding with OpenAI Reasoning Models | onekq/webapp1k | 2024 |
| 3 | o1-preview | 0.652 | A Case Study of Web App Coding with OpenAI Reasoning Models | onekq/webapp1k | 2024 |
| 4 | gpt-4o-2024-08-06 | 0.531 | A Case Study of Web App Coding with OpenAI Reasoning Models | onekq/webapp1k | 2024 |
| 5 | deepseek-v2.5 | 0.49 | A Case Study of Web App Coding with OpenAI Reasoning Models | onekq/webapp1k | 2024 |
| 6 | mistral-large-2 | 0.449 | A Case Study of Web App Coding with OpenAI Reasoning Models | onekq/webapp1k | 2024 |