Countering the Influence of Essay Length in Neural Essay Scoring
Previous work has shown that automated essay scoring systems, in particular machine learning-based systems, are not capable of assessing the quality of essays, but are relying on essay length, a factor irrelevant to writing proficiency. In this work, we first show that state-of-the-art systems, recent neural essay scoring systems, might be also influenced by the correlation between essay length and scores in a standard dataset. In our evaluation, a very simple neural model shows the state-of-the-art performance on the standard dataset. To consider essay content without taking essay length into account, we introduce a simple neural model assessing the similarity of content between an input essay and essays assigned different scores. This neural model achieves performance comparable to the state of the art on a standard dataset as well as on a second dataset. Our findings suggest that neural essay scoring systems should consider the characteristics of datasets to focus on text quality.
Code (1)
Tasks
Automated Essay ScoringSimilar Papers 제목 키워드 기반
From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models
Recent advances in Large Language Models (LLMs) have substantially transformed Automated Essay Scoring (AES), yet the internal mechanisms underlying LLM-based scoring remain poorly understood. In this work, we systematic…
Dimensionality ReductionAutomated Essay ScoringNeural Multi-task Learning in Automated Assessment
Grammatical error detection and automated essay scoring are two tasks in the area of automated assessment. Traditionally these tasks have been treated independently with different machine learning models and features use…
Automated Essay ScoringBIG-bench Machine LearningGrammatical Error DetectionMulti-Task LearningToward Educator-focused Automated Scoring Systems for Reading and Writing
This paper presents methods for improving automated essay scoring with techniques that address the computational trade-offs of self-attention and document length. To make Automated Essay Scoring (AES) more useful to prac…
Automated Essay ScoringSentenceSentence EmbeddingsTransfer LearningCan generative AI figure out figurative language? The influence of idioms on essay scoring by ChatGPT, Gemini, and Deepseek
The developments in Generative AI technologies have paved the way for numerous innovations in different fields. Recently, Generative AI has been proposed as a competitor to AES systems in evaluating student essays automa…
Long Context Automated Essay Scoring with Language Models
Transformer-based language models are architecturally constrained to process text of a fixed maximum length. Essays written by higher-grade students frequently exceed the maximum allowed length for many popular open-sour…
Automated Essay Scoring