On the Impacts of Contexts on Repository-Level Code Generation

2024-06-17 · Nam Le Hai, Dung Manh Nguyen, Nghi D. Q. Bui

CodeLLMs have gained widespread adoption for code generation tasks, yet their capacity to handle repository-level code generation with complex contextual dependencies remains underexplored. Our work underscores the critical importance of leveraging repository-level contexts to generate executable and functionally correct code. We present RepoExec, a novel benchmark designed to evaluate repository-level code generation, with a focus on three key aspects: executability, functional correctness through comprehensive test case generation, and accurate utilization of cross-file contexts. Our study examines a controlled scenario where developers specify essential code dependencies (contexts), challenging models to integrate them effectively. Additionally, we introduce an instruction-tuned dataset that enhances CodeLLMs' ability to leverage dependencies, along with a new metric, Dependency Invocation Rate (DIR), to quantify context utilization. Experimental results reveal that while pretrained LLMs demonstrate superior performance in terms of correctness, instruction-tuned models excel in context utilization and debugging capabilities. RepoExec offers a comprehensive evaluation framework for assessing code functionality and alignment with developer intent, thereby advancing the development of more reliable CodeLLMs for real-world applications. The dataset and source code are available at https://github.com/FSoft-AI4Code/RepoExec.

📄 PDF Abstract BibTeX arXiv:2406.11927

Code (1)

FSoft-AI4Code/RepoExec 공식 구현

Tasks

Code Generation

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Repoformer: Selective Retrieval for Repository-Level Code Completion

2024-03-15 · Di wu, Wasi Uddin Ahmad, Dejiao Zhang, Murali Krishna Ramanathan 외

Recent advances in retrieval-augmented generation (RAG) have initiated a new era in repository-level code completion. However, the invariable use of retrieval in existing methods exposes issues in both efficiency and rob…

Code Completion RAG Retrieval Retrieval-augmented Generation+1

Impact-driven Context Filtering For Cross-file Code Completion

2025-08-08 · Yanzhou Li, Shangqing Liu, Kangjie Chen, Tianwei Zhang 외 arxiv

Retrieval-augmented generation (RAG) has recently demonstrated considerable potential for repository-level code completion, as it integrates cross-file knowledge with in-file preceding code to provide comprehensive conte…

Computational Efficiency Code Completion

A Review of Repository Level Prompting for LLMs

2023-12-15 · Douglas Schonholtz

As coding challenges become more complex, recent advancements in Large Language Models (LLMs) have led to notable successes, such as achieving a 94.6\% solve rate on the HumanEval benchmark. Concurrently, there is an inc…

Code Completion Code Generation HumanEval Retrieval

ContextSniper: AntTrail's Token-Efficient Code Memory for Repository-Level Program Repair

2026-07-02 · Chiwang Luk, Matin Mohammad Najafi, Zhifeng Jia, Wei Yang 외 arxiv

Large language model agents can repair real repository issues, but they often spend large context budgets on whole-file reads, broad searches, and long terminal outputs where useful evidence is mixed with irrelevant code…

Program Repair

Hierarchical Embedding Fusion for Retrieval-Augmented Code Generation

2026-02-04 · Nikita Sorokin, Ivan Sedykh, Valentin Malykh arxiv

Retrieval-augmented code generation often conditions the decoder on large retrieved code snippets. This ties online inference cost to repository size and introduces noise from long contexts. We present Hierarchical Embed…

Code Generation Code Completion