Code Generation on CoNaLa

14개 결과 · ⬇ CSV · JSON

BLEU

Rank	Model	BLEU	Exact Match Accuracy	Extra Training Data	Paper	Code	Year
1	PanGu-Coder-FT-I	44.32	–		Fine-Tuning Large Language Models for Answering Programming Questions with Code Snippets		2023
2	RoBERTaMarian	35.74	13.8		Leveraging pre-trained language models for code generation	AhmedSSoliman/Leveraging-Pretrained-Language-Models-for-Code-Generation	2024
3	MarianCG	34.43	10.2		MarianCG: a code generation transformer model inspired by machine translation	AhmedSSoliman/MarianCG-NL-to-Code	2022
4	TranX + BERT w/mined	34.2	5.8		The impact of lexical and grammatical processing on generating code from natural language	codegenfactors/BertranX · codegenfact/BertranX	2022
5	BERT + TAE	33.41	–	✓	Code Generation from Natural Language with Less Prior and More Monolingual Data	borealisai/code-gen-tae	2021
6	BERTMarian	32.46	12.40		Leveraging pre-trained language models for code generation	AhmedSSoliman/Leveraging-Pretrained-Language-Models-for-Code-Generation	2024
7	External Knowledge With API + Reranking	32.26	–	✓	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation	neulab/external-knowledge-codegen · zorazrw/multilingual-conala	2020
8	External Knowledge With API	30.69	–	✓	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation	neulab/external-knowledge-codegen · zorazrw/multilingual-conala	2020
9	BART W/ Mined	30.55	–		Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation	gabeorlanski/stackoverflow-encourages-cheating	2021
10	ELECTRAMarian	30.18	10.0		Leveraging pre-trained language models for code generation	AhmedSSoliman/Leveraging-Pretrained-Language-Models-for-Code-Generation	2024
11	Reranker	30.11	2.8		Reranking for Neural Semantic Parsing		2019
12	LUKEMarian	29.83	7.6		Leveraging pre-trained language models for code generation	AhmedSSoliman/Leveraging-Pretrained-Language-Models-for-Code-Generation	2024
13	BART Base	26.24	–		Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation	gabeorlanski/stackoverflow-encourages-cheating	2021
14	TranX	24.30	–		TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation	pcyin/tranX · DeepLearnXMU/CG-RL · yuxiang2/CoNaLa · +1	2018