Code Generation on APPS

19개 결과 · ⬇ CSV · JSON

Introductory Pass@1

Rank	Model	Introductory Pass@1	Interview Pass@1	Competition Pass@1	Competition Pass@any	Interview Pass@any	Introductory Pass@any	Competition Pass@5	Interview Pass@5	Paper	Code	Year
1	LPW (GPT-4o)	87.2	65.2	34.8	–	–	–	–	–	Planning-Driven Programming: A Large Language Model Programming Workflow	you68681/lpw	2024
2	MoTCoder-32B-V1.5	68.44	44.49	27.84	–	–	–	–	–	MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks	dvlab-research/motcoder	2023
3	MoTCoder-7B-V1.5	54.26	32.63	21.18	–	–	–	–	–	MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks	dvlab-research/motcoder	2023
4	code-davinci-002 175B (CodeT)	47.3%	14.3%	6.2%	–	–	–	–	–	CodeT: Code Generation with Generated Tests	microsoft/codet	2022
5	deepseek-ai/deepseek-coder-6.7b-instruct	33.80	19.70	11.09	–	–	–	–	–	DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence	deepseek-ai/DeepSeek-Coder	2024
6	code-davinci-002 175B	31.92	–	–	–	–	–	–	–	CodeT: Code Generation with Generated Tests	microsoft/codet	2022
7	CodeChain+WizardCoder-15b	29.3%	6.4%	2.5%	14.5%	25.4%	60.9%	–	–	CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules	SalesforceAIResearch/CodeChain	2023
8	WizardCoder-15b	26.29	7.49	3.75	–	–	–	–	–	CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules	SalesforceAIResearch/CodeChain	2023
9	CodeSim (GPT4)	26.04	4.21	0.81	–	–	–	–	–	CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging	kagnlp/CodeGenerator	2025
10	CodeRL+CodeT5	20	13.5	33.3	–	–	–	–	–	CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning	salesforce/codet5 · salesforce/coderl	2022
11	GPT-J 6B (Finetuned)	6.77%	1.80%	0.69%	15.70%	14.33%	38.10%	2.36%	4.48%	CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning	salesforce/codet5 · salesforce/coderl	2022
12	Codex 12B (Raw)	5.60%	1.00%	0.50%	13.51%	13.15%	35.20%	1.00%	1.73%	Evaluating Large Language Models Trained on Code	THUDM/CodeGeeX · ncoop57/gpt-code-clippy · codedotal/gpt-code-clippy · +10	2021
13	GPT-Neo 2.7B (Finetuned)	4.14%	0.14%	0.02%	3.32%	3.70%	25.02%	0.09%	0.51%	CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning	salesforce/codet5 · salesforce/coderl	2022
14	GPT-Neo 2.7B	3.90%	0.57%	0.00%	11.40%	9.83%	27.90%	0.00%	0.80%	Measuring Coding Challenge Competence With APPS	ncoop57/gpt-code-clippy · codedotal/gpt-code-clippy · hendrycks/apps	2021
14	GPT2 1.5B (Finetuned)	3.90%	0.57%	0.00%	0.0%	0.80%	5.50%	0.00%	0.80%	CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning	salesforce/codet5 · salesforce/coderl	2022
16	MapCoder APPS-150-cherrypicked (GPT-4)	1.30%	0.70%	0.00%	8.80%	9.27%	25.00%	0.00%	1.03%	MapCoder: Multi-Agent Code Generation for Competitive Problem Solving	md-ashraful-pramanik/mapcoder · Luoji-zju/Agents4PLC_release	2024
17	AlphaCode 1B Filtered from 50000	–	–	–	22.0	–	–	–	–	Competition-Level Code Generation with AlphaCode	google-deepmind/code_contests · deepmind/code_contests	2022
18	AlphaCode 1B	–	–	–	7.75%	9.66%	20.36%	7.75%	9.66%	Competition-Level Code Generation with AlphaCode	google-deepmind/code_contests · deepmind/code_contests	2022
19	SolidCoder 자동 추출	–	–	–	–	–	–	–	–	SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution		2026