Cognitively Aided Zero-Shot Automatic Essay Grading
Automatic essay grading (AEG) is a process in which machines assign a grade to an essay written in response to a topic, called the prompt. Zero-shot AEG is when we train a system to grade essays written to a new prompt which was not present in our training data. In this paper, we describe a solution to the problem of zero-shot automatic essay grading, using cognitive information, in the form of gaze behaviour. Our experiments show that using gaze behaviour helps in improving the performance of AEG systems, especially when we provide a new essay written in response to a new prompt for scoring, by an average of almost 5 percentage points of QWK.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
LCES: Zero-shot Automated Essay Scoring via Pairwise Comparisons Using Large Language Models
Recent advances in large language models (LLMs) have enabled zero-shot automated essay scoring (AES), providing a promising way to reduce the cost and effort of essay scoring in comparison with manual grading. However, m…
Automated Essay ScoringComputational EfficiencyExploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation
Individual feedback can help students improve their essay writing skills. However, the manual effort required to provide such feedback limits individualization in practice. Automatically-generated essay feedback may serv…
Automated Essay ScoringIs GPT-4 Alone Sufficient for Automated Essay Scoring?: A Comparative Judgment Approach Based on Rater Cognition
Large Language Models (LLMs) have shown promise in Automated Essay Scoring (AES), but their zero-shot and few-shot performance often falls short compared to state-of-the-art models and human raters. However, fine-tuning …
Automated Essay ScoringUnleashing Large Language Models' Proficiency in Zero-shot Essay Scoring
Advances in automated essay scoring (AES) have traditionally relied on labeled essays, requiring tremendous cost and expertise for their acquisition. Recently, large language models (LLMs) have achieved great success in …
Automated Essay ScoringStructured Prompting for Arabic Essay Proficiency: A Trait-Centric Evaluation Approach
This paper presents a novel prompt engineering framework for trait specific Automatic Essay Scoring (AES) in Arabic, leveraging large language models (LLMs) under zero-shot and few-shot configurations. Addressing the sca…
Prompt Engineering