Text-to-Code Generation on CodeXGLUE - CONCODE
BLEU
- 2021-02-09 — CodeGPT-adapted: BLEU 32.79
- 2021-09-02 — CodeT5: BLEU 41.48
| Rank | Model | BLEU | CodeBLEU | EM | Paper | Code | Year |
|---|---|---|---|---|---|---|---|
| 1 | CodeT5 | 41.48 | 44.10 | 22.70 | CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation | salesforce/codet5 · salesforce/coderl · awsm-research/vulrepair · +2 | 2021 |
| 2 | CodeGPT-adapted | 32.79 | 27.74 | 20.1 | CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation | microsoft/CodeXGLUE · facebookresearch/CodeGen · sberbank-ai/fusion_brain_aij2021 · +4 | 2021 |
| 3 | CodeT5 | 41.48 | 44.10 | 22.70 | CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation | salesforce/codet5 · salesforce/coderl · awsm-research/vulrepair · +2 | 2021 |
| 4 | CodeGPT-adapted | 32.79 | 27.74 | 20.1 | CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation | microsoft/CodeXGLUE · facebookresearch/CodeGen · sberbank-ai/fusion_brain_aij2021 · +4 | 2021 |