Context-Augmented Code Generation Using Programming Knowledge Graphs
Large Language Models (LLMs) excel at code generation but struggle with complex problems. Retrieval-Augmented Generation (RAG) mitigates this issue by integrating external knowledge, yet retrieval models often miss relevant context, and generation models hallucinate with irrelevant data. We propose Programming Knowledge Graph (PKG) for semantic representation and fine-grained retrieval of code and text. Our approach enhances retrieval precision through tree pruning and mitigates hallucinations via a re-ranking mechanism that integrates non-RAG solutions. Structuring external data into finer-grained nodes improves retrieval granularity. Evaluations on HumanEval and MBPP show up to 20% pass@1 accuracy gains and a 34% improvement over baselines on MBPP. Our findings demonstrate that our proposed PKG approach along with re-ranker effectively address complex problems while maintaining minimal negative impact on solutions that are already correct without RAG. The replication package is published at https://github.com/iamshahd/ProgrammingKnowledgeGraph
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsCode GenerationResults from the Paper
| Rank | Task | Dataset | Model | Metrics |
|---|---|---|---|---|
| #89 | Code Generation | MBPP | Programming | Accuracy: 34 |
Similar Papers 제목 키워드 기반
Context-Augmented Code Generation Using Programming Knowledge Graphs
Large Language Models (LLMs) and Code-LLMs (CLLMs) have significantly improved code generation, but, they frequently face difficulties when dealing with challenging and complex problems. Retrieval-Augmented Generation (R…
Code GenerationHumanEvalKnowledge Graphsmbpp+4CodeGRAG: Bridging the Gap between Natural Language and Programming Language via Graphical Retrieval Augmented Generation
Utilizing large language models to generate codes has shown promising meaning in software development revolution. Despite the intelligence shown by the large language models, their specificity in code generation can stil…
Code GenerationInformativenessLanguage ModellingLarge Language Model+3Contextual Augmented Multi-Model Programming (CAMP): A Hybrid Local-Cloud Copilot Framework
The advancements in cloud-based Large Languages Models (LLMs) have revolutionized AI-assisted programming. However, their integration into certain local development environments like ones within the Apple software ecosys…
Code CompletionRAGRetrieval-augmented GenerationCodeRAG-Bench: Can Retrieval Augment Code Generation?
While language models (LMs) have proven remarkably adept at generating code, many programs are challenging for LMs to generate using their parametric knowledge alone. Providing external contexts such as library documenta…
Code GenerationRAGRetrievalRetrieval-augmented GenerationLeveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generation
This paper presents the use of Retrieval Augmented Generation (RAG) to improve the feedback generated by Large Language Models for programming tasks. For this purpose, corresponding lecture recordings were transcribed an…
Language ModellingLarge Language ModelRAGRetrieval+1