HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis
As the use of Large Language Models (LLMs) in text generation tasks proliferates, concerns arise over their potential to compromise academic integrity. The education sector currently tussles with distinguishing student-authored homework assignments from AI-generated ones. This paper addresses the challenge by introducing HowkGPT, designed to identify homework assignments generated by AI. HowkGPT is built upon a dataset of academic assignments and accompanying metadata [17] and employs a pretrained LLM to compute perplexity scores for student-authored and ChatGPT-generated responses. These scores then assist in establishing a threshold for discerning the origin of a submitted assignment. Given the specificity and contextual nature of academic work, HowkGPT further refines its analysis by defining category-specific thresholds derived from the metadata, enhancing the precision of the detection. This study emphasizes the critical need for effective strategies to uphold academic integrity amidst the growing influence of LLMs and provides an approach to ensuring fair and accurate grading in educational institutions.
Code (0)
등록된 구현이 없습니다.
Tasks
SpecificityText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Perception, performance, and detectability of conversational artificial intelligence across 32 university courses
The emergence of large language models has led to the development of powerful tools such as ChatGPT that can produce text indistinguishable from human-generated work. With the increasing accessibility of such technology,…
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?
Recent advances in natural language processing (NLP) have led to the development of large language models (LLMs) such as ChatGPT. This paper proposes a methodology for developing and evaluating ChatGPT detectors for Fren…
Adversarial TextLanguage ModelingLanguage ModellingWaiting, Banning, and Embracing: An Empirical Analysis of Adapting Policies for Generative AI in Higher Education
Generative AI tools such as ChatGPT have recently gained significant attention in higher education. This study aims to understand how universities establish policies regarding the use of AI tools and explore the factors …
ChatGPT as a Math Questioner? Evaluating ChatGPT on Generating Pre-university Math Questions
Mathematical questioning is crucial for assessing students problem-solving skills. Since manually creating such questions requires substantial effort, automatic methods have been explored. Existing state-of-the-art model…
Arithmetic ReasoningMathQuestion AnsweringClassification of Human- and AI-Generated Texts: Investigating Features for ChatGPT
Recently, generative AIs like ChatGPT have become available to the wide public. These tools can for instance be used by students to generate essays or whole theses. But how does a teacher know whether a text is written b…