Code Search 벤치마크
Code Search on CodeSearchNet
Overall
- 2020-02-19 — CodeBERT: Overall 76.0
- 2020-09-17 — GraphCodeBERT: Overall 77.4
- 2022-01-24 — cpt-code M: Overall 93.5
| Rank | Model | Overall | Go | Ruby | Python | Java | JS | PHP | Paper | Code | Year |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | cpt-code M | 93.5 | 97.5 | 85.5 | 99.9 | 94.4 | 86.5 | 97.2 | Text and Code Embeddings by Contrastive Pre-Training | openmatch/coco-dr | 2022 |
| 2 | cpt-code S | 93.4 | 97.7 | 86.3 | 99.8 | 94.0 | 86.0 | 96.7 | Text and Code Embeddings by Contrastive Pre-Training | openmatch/coco-dr | 2022 |
| 3 | CodeT5+ 770M | 77.4 | 92.7 | 78.0 | 75.8 | 76.2 | 71.3 | 70.1 | CodeT5+: Open Code Large Language Models for Code Understanding and Generation | salesforce/codet5 · leiluk1/codesearcher | 2023 |
| 3 | GraphCodeBERT | 77.4 | 84.1 | 73.2 | 87.9 | 75.7 | 71.1 | 72.5 | GraphCodeBERT: Pre-training Code Representations with Data Flow | microsoft/CodeBERT | 2020 |
| 5 | CodeT5+ 220M | 77.1 | 92.4 | 77.7 | 75.6 | 76.1 | 70..8 | 69.8 | CodeT5+: Open Code Large Language Models for Code Understanding and Generation | salesforce/codet5 · leiluk1/codesearcher | 2023 |
| 6 | CodeBERT | 76.0 | 69.3 | 70.6 | 84.0 | 86.8 | 74.8 | 70.6 | CodeBERT: A Pre-Trained Model for Programming and Natural Languages | salesforce/codet5 · microsoft/CodeBERT · graykode/commit-autosuggestions · +6 | 2020 |