Evaluating the Quality of Code Comments Generated by Large Language Models for Novice Programmers

2024-09-22 · Aysa Xuemo Fan, Arun Balajiee Lekshmi Narayanan, Mohammad Hassany, Jiaze Ke

Large Language Models (LLMs) show promise in generating code comments for novice programmers, but their educational effectiveness remains under-evaluated. This study assesses the instructional quality of code comments produced by GPT-4, GPT-3.5-Turbo, and Llama2, compared to expert-developed comments, focusing on their suitability for novices. Analyzing a dataset of ``easy'' level Java solutions from LeetCode, we find that GPT-4 exhibits comparable quality to expert comments in aspects critical for beginners, such as clarity, beginner-friendliness, concept elucidation, and step-by-step guidance. GPT-4 outperforms Llama2 in discussing complexity (chi-square = 11.40, p = 0.001) and is perceived as significantly more supportive for beginners than GPT-3.5 and Llama2 with Mann-Whitney U-statistics = 300.5 and 322.5, p = 0.0017 and 0.0003). This study highlights the potential of LLMs for generating code comments tailored to novice programmers.

📄 PDF Abstract BibTeX arXiv:2409.14368

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Refunds@Expedia|||How do I get a full refund from Expedia? “How do I get a full refund from Expedia? How do I get a full refund from Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Quick Help &…

{Dispute@FaQ-s}How to file a dispute with Expedia? How to file a dispute with Expedia? To file a complaint against Expedia, first try contacting their customer service directly. You can reach them by phone at…

15 Ways to Contact How can i speak to someone at Delta Airlines 설명 없음

Attention 설명 없음

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Weight Decay 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks

2025-04-28 · Kang Yang, XinJun Mao, Shangwen Wang, Yanlin Wang 외

Pre-trained code models rely heavily on high-quality pre-training data, particularly human-written reference comments that bridge code and natural language. However, these comments often become outdated as software evolv…

Code Search Code Summarization

evalSmarT: An LLM-Based Framework for Evaluating Smart Contract Generated Comments

2025-07-28 · Fatou Ndiaye Mbodji arxiv

Smart contract comment generation has gained traction as a means to improve code comprehension and maintainability in blockchain systems. However, evaluating the quality of generated comments remains a challenge. Traditi…

Revisiting the Role of Natural Language Code Comments in Code Translation

2026-01-23 · Monika Gupta, Ajay Meena, Anamitra Roy Choudhury, Vijay Arya 외 arxiv

The advent of large language models (LLMs) has ushered in a new era in automated code translation across programming languages. Since most code-specific LLMs are pretrained on well-commented code from large repositories …

Code Translation

SecureReviewer: Enhancing Large Language Models for Secure Code Review through Secure-aware Fine-tuning

2025-10-30 · Fang Liu, Simiao Liu, Yinghao Zhu, Xiaoli Lian 외 arxiv

Identifying and addressing security issues during the early phase of the development lifecycle is critical for mitigating the long-term negative impacts on software systems. Code review serves as an effective practice th…

An Exploratory Study on LLM-Generated Code and Comments in Code Repositories

2026-07-02 · Yongyi Ji, Jiaji Wang, Yi Zhou, Fuxiang Chen 외 arxiv

The use of LLMs in software development has become increasingly widespread on tasks such as code generation and summarization. Reports from large technology companies showed that around 20% to 30% of their code are gener…

Code Generation