paper-with-me

Code Generation 벤치마크

Code Generation on CoNaLa-Ext

6개 결과 · ⬇ CSV · JSON

BLEU

18.85 22.97 27.09 31.2 35.32 2018-10 2026-09 TranX — 18.85 (2018-10-05) Reranker — 19.85 (2019-07-01) External Knowledge With API + Reranking — 20.54 (2020-04-20) External Knowledge With API — 20.37 (2020-04-20) BART W/ Mined — 35.32 (2021-06-08) BART Base — 34.35 (2021-06-08) TranX — 18.85 (2018-10-05) Reranker — 19.85 (2019-07-01) External Knowledge With API + Reranking — 20.54 (2020-04-20) BART W/ Mined — 35.32 (2021-06-08)
RankModel BLEU PaperCodeYear
1 BART W/ Mined 35.32 Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation gabeorlanski/stackoverflow-encourages-cheating 2021
2 BART Base 34.35 Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation gabeorlanski/stackoverflow-encourages-cheating 2021
3 External Knowledge With API + Reranking 20.54 Incorporating External Knowledge through Pre-training for Natural Language to Code Generation neulab/external-knowledge-codegen · zorazrw/multilingual-conala 2020
4 External Knowledge With API 20.37 Incorporating External Knowledge through Pre-training for Natural Language to Code Generation neulab/external-knowledge-codegen · zorazrw/multilingual-conala 2020
5 Reranker 19.85 Reranking for Neural Semantic Parsing 2019
6 TranX 18.85 TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation pcyin/tranX · DeepLearnXMU/CG-RL · yuxiang2/CoNaLa · +1 2018
1–6 / 6 페이지당 10 20 50 100