Class-level Code Generation
1개 벤치마크 · 논문 7편 · 이 태스크의 논문 보기 →
Benchmarks
RESIDE TestSet-S
Most implemented
pix2code: Generating Code from a Graphical User Interface Screenshot
ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation
AutoCoder: Enhancing Code Large Language Model with \textsc{AIEV-Instruct}
Papers
ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
LLMs have achieved strong results on both function-level code synthesis and repository-level code modification, yet a capability that falls between these two extremes -- compositional code creation, i.e., building a comp…
Class-level Code GenerationBeyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation
Large language models (LLMs) have demonstrated strong performance on function-level code generation benchmarks, yet real-world software development increasingly demands class-level implementations that integrate multiple…
Class-level Code GenerationA Large-scale Class-level Benchmark Dataset for Code Generation with LLMs
Recent advancements in large language models (LLMs) have demonstrated promising capabilities in code generation tasks. However, most existing benchmarks focus on isolated functions and fail to capture the complexity of r…
BenchmarkingClass-level Code GenerationCode GenerationAutoCoder: Enhancing Code Large Language Model with \textsc{AIEV-Instruct}
We introduce AutoCoder, the first Large Language Model to surpass GPT-4 Turbo (April 2024) and GPT-4o in pass@1 on the Human Eval benchmark test ($\mathbf{90.9\%}$ vs. $\mathbf{90.2\%}$). In addition, AutoCoder offers a …
Class-level Code GenerationCode CompletionCode GenerationCode Repair+8Class-Level Code Generation from Natural Language Using Iterative, Tool-Enhanced Reasoning over Repository
LLMs have demonstrated significant potential in code generation tasks, achieving promising results at the function or statement level across various benchmarks. However, the complexities associated with creating code art…
Class-level Code GenerationCode GenerationNavigateClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation
In this work, we make the first attempt to evaluate LLMs in a more challenging code generation scenario, i.e. class-level code generation. We first manually construct the first class-level code generation benchmark Class…
Class-level Code GenerationCode GenerationHumanEval