Analysis on LLMs Performance for Code Summarization
Code summarization aims to generate concise natural language descriptions for source code. Deep learning has been used more and more recently in software engineering, particularly for tasks like code creation and summarization. Specifically, it appears that the most current Large Language Models with coding perform well on these tasks. Large Language Models (LLMs) have significantly advanced the field of code summarization, providing sophisticated methods for generating concise and accurate summaries of source code. This study aims to perform a comparative analysis of several open-source LLMs, namely LLaMA-3, Phi-3, Mistral, and Gemma. These models' performance is assessed using important metrics such as BLEU\textsubscript{3.1} and ROUGE\textsubscript{3.2}. Through this analysis, we seek to identify the strengths and weaknesses of each model, offering insights into their applicability and effectiveness in code summarization tasks. Our findings contribute to the ongoing development and refinement of LLMs, supporting their integration into tools that enhance software development and maintenance processes.
Code (0)
등록된 구현이 없습니다.
Tasks
Code SummarizationSimilar Papers 제목 키워드 기반
ETF: An Entity Tracing Framework for Hallucination Detection in Code Summaries
Recent advancements in large language models (LLMs) have significantly enhanced their ability to understand both natural language and code, driving their use in tasks like natural language-to-code (NL2Code) and code summ…
Code SummarizationHallucinationChain-of-Descriptions: Improving Code LLMs for VHDL Code Generation and Summarization
Large Language Models (LLMs) have become widely used across diverse NLP tasks and domains, demonstrating their adaptability and effectiveness. In the realm of Electronic Design Automation (EDA), LLMs show promise for tas…
Code GenerationHierarchical Repository-Level Code Summarization for Business Applications Using Local LLMs
In large-scale software development, understanding the functionality and intent behind complex codebases is critical for effective development and maintenance. While code summarization has been widely studied, existing m…
Code SummarizationOn Context Utilization in Summarization with Large Language Models
Large language models (LLMs) excel in abstractive summarization tasks, delivering fluent and pertinent summaries. Recent advancements have extended their capabilities to handle long-input contexts, exceeding 100k tokens.…
Abstractive Text SummarizationPositionQuestion AnsweringAutomatic Semantic Augmentation of Language Model Prompts (for Code Summarization)
Large Language Models (LLM) are a new class of computation engines, "programmed" via prompt engineering. We are still learning how to best "program" these LLMs to help developers. We start with the intuition that develop…
Code SummarizationInformation RetrievalLanguage ModelingLanguage Modelling+2