Code Completion 벤치마크
Code Completion on CodeXGLUE - PY150
| Rank | Model | Accuracy (token-level) | EM (line-level) | Edit Sim (line-level) | Paper | Code | Year |
|---|---|---|---|---|---|---|---|
| 1 | CodeGPT-adapted | 75.11 | 39.65 | 69.84 | CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation | microsoft/CodeXGLUE · facebookresearch/CodeGen · sberbank-ai/fusion_brain_aij2021 · +4 | 2021 |
| 2 | CodeT5+ 770M | – | 44.86 | 74.22 | CodeT5+: Open Code Large Language Models for Code Understanding and Generation | salesforce/codet5 · leiluk1/codesearcher | 2023 |
| 3 | CodeT5+ 220M | – | 43.42 | 73.69 | CodeT5+: Open Code Large Language Models for Code Understanding and Generation | salesforce/codet5 · leiluk1/codesearcher | 2023 |