Testing the Effect of Code Documentation on Large Language Model Code Understanding
Large Language Models (LLMs) have demonstrated impressive abilities in recent years with regards to code generation and understanding. However, little work has investigated how documentation and other code properties affect an LLM's ability to understand and generate code or documentation. We present an empirical analysis of how underlying properties of code or documentation can affect an LLM's capabilities. We show that providing an LLM with "incorrect" documentation can greatly hinder code understanding, while incomplete or missing documentation does not seem to significantly affect an LLM's ability to understand code.
Code (0)
등록된 구현이 없습니다.
Tasks
Code GenerationLanguage ModelingLanguage ModellingLarge Language ModelSimilar Papers 제목 키워드 기반
A Study on the Improvement of Code Generation Quality Using Large Language Models Leveraging Product Documentation
Research on using Large Language Models (LLMs) in system development is expanding, especially in automated code and test generation. While E2E testing is vital for ensuring application quality, most test generation resea…
Code GenerationToolFuzz -- Automated Agent Tool Testing
Large Language Model (LLM) Agents leverage the advanced reasoning capabilities of LLMs in real-world applications. To interface with an environment, these agents often rely on tools, such as web search or database APIs. …
Large Language ModelPrompt Engineeringsoftware testingExploring Large Language Models for Code Explanation
Automating code documentation through explanatory text can prove highly beneficial in code understanding. Large Language Models (LLMs) have made remarkable strides in Natural Language Processing, especially within softwa…
Code GenerationCode SummarizationBuilding A Coding Assistant via the Retrieval-Augmented Language Model
Pretrained language models have shown strong effectiveness in code-related tasks, such as code retrieval, code generation, code summarization, and code completion tasks. In this paper, we propose COde assistaNt viA retri…
Code CompletionCode GenerationCode SummarizationLanguage Modeling+3CIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Models
Software architecture documentation is essential for system comprehension, yet it is often unavailable or incomplete. While recent LLM-based techniques can generate documentation from code, they typically address local a…