Code Completion 벤치마크
Code Completion on CodeXGLUE - Github Java Corpus
| Rank | Model | Accuracy (token-level) | EM (line-level) | Edit Sim (line-level) | Paper | Code | Year |
|---|---|---|---|---|---|---|---|
| 1 | CodeGPT-adapted | 77.13 | 26.43 | 63.03 | CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation | microsoft/CodeXGLUE · facebookresearch/CodeGen · sberbank-ai/fusion_brain_aij2021 · +4 | 2021 |
| 2 | CodeT5+ 770M | – | 37.90 | 72.25 | CodeT5+: Open Code Large Language Models for Code Understanding and Generation | salesforce/codet5 · leiluk1/codesearcher | 2023 |
| 3 | CodeT5+ 220M | – | 35.17 | 69.48 | CodeT5+: Open Code Large Language Models for Code Understanding and Generation | salesforce/codet5 · leiluk1/codesearcher | 2023 |