Code Generation 벤치마크
Code Generation on Shellcode_IA32
BLEU-4
- 2021-04-27 — LSTM-based Sequence to Sequence: BLEU-4 62.97
- 2022-02-08 — CodeBERT: BLEU-4 91.7
| Rank | Model | BLEU-4 | Exact Match Accuracy | Paper | Code | Year |
|---|---|---|---|---|---|---|
| 1 | CodeBERT | 91.70 | 89.75 | Can We Generate Shellcodes via Natural Language? An Empirical Study | dessertlab/Shellcode_IA32 | 2022 |
| 2 | Seq2Seq with Attention | 90.03 | 82.92 | Can We Generate Shellcodes via Natural Language? An Empirical Study | dessertlab/Shellcode_IA32 | 2022 |
| 3 | LSTM-based Sequence to Sequence | 62.97 | 51.55 | Shellcode_IA32: A Dataset for Automatic Shellcode Generation | dessertlab/Shellcode_IA32 | 2021 |