Inappropriate Benefits and Identification of ChatGPT Misuse in Programming Tests: A Controlled Experiment
While ChatGPT may help students to learn to program, it can be misused to do plagiarism, a breach of academic integrity. Students can ask ChatGPT to complete a programming task, generating a solution from other people's work without proper acknowledgment of the source(s). To help address this new kind of plagiarism, we performed a controlled experiment measuring the inappropriate benefits of using ChatGPT in terms of completion time and programming performance. We also reported how to manually identify programs aided with ChatGPT (via student behavior while using ChatGPT) and student perspective of ChatGPT (via a survey). Seventeen students participated in the experiment. They were asked to complete two programming tests. They were divided into two groups per the test: one group should complete the test without help while the other group should complete it with ChatGPT. Our study shows that students with ChatGPT complete programming tests two times faster than those without ChatGPT, though their programming performance is comparable. The generated code is highly efficient and uses complex data structures like lists and dictionaries. Based on the survey results, ChatGPT is recommended to be used as an assistant to complete programming tasks and other general assignments. ChatGPT will be beneficial as a reference as other search engines do. Logical and critical thinking are needed to validate the result presented by ChatGPT.
Code (0)
등록된 구현이 없습니다.
Tasks
SurveySimilar Papers 제목 키워드 기반
Plagiarism and AI Assistance Misuse in Web Programming: Unfair Benefits and Characteristics
In programming education, plagiarism and misuse of artificial intelligence (AI) assistance are emerging issues. However, not many relevant studies are focused on web programming. We plan to develop automated tools to hel…
An Investigation into Misuse of Java Security APIs by Large Language Models
The increasing trend of using Large Language Models (LLMs) for code generation raises the question of their capability to generate trustworthy code. While many researchers are exploring the utility of code generation for…
Code GenerationCritical Role of Artificially Intelligent Conversational Chatbot
Artificially intelligent chatbot, such as ChatGPT, represents a recent and powerful advancement in the AI domain. Users prefer them for obtaining quick and precise answers, avoiding the usual hassle of clicking through m…
ChatbotAnalysis of ChatGPT on Source Code
This paper explores the use of Large Language Models (LLMs) and in particular ChatGPT in programming, source code analysis, and code generation. LLMs and ChatGPT are built using machine learning and artificial intelligen…
Code GenerationChatGPT's Potential in Cryptography Misuse Detection: A Comparative Analysis with Static Analysis Tools
The correct adoption of cryptography APIs is challenging for mainstream developers, often resulting in widespread API misuse. Meanwhile, cryptography misuse detectors have demonstrated inconsistent performance and remain…
Prompt Engineering