Automated Scoring: Beyond Natural Language Processing
In this position paper, we argue that building operational automated scoring systems is a task that has disciplinary complexity above and beyond standard competitive shared tasks which usually involve applying the latest machine learning techniques to publicly available data in order to obtain the best accuracy. Automated scoring systems warrant significant cross-discipline collaboration of which natural language processing and machine learning are just two of many important components. Such systems have multiple stakeholders with different but valid perspectives that can often times be at odds with each other. Our position is that it is essential for us as NLP researchers to understand and incorporate these perspectives in our research and work towards a mutually satisfactory solution in order to build automated scoring systems that are accurate, fair, unbiased, and useful.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningPositionvalidSimilar Papers 제목 키워드 기반
G-SciEdBERT: A Contextualized LLM for Science Assessment Tasks in German
The advancement of natural language processing has paved the way for automated scoring systems in various languages, such as German (e.g., German BERT [G-BERT]). Automatically scoring written responses to science questio…
Language ModellingLarge Language ModelData Augmentation for Automated Essay Scoring using Transformer Models
Automated essay scoring is one of the most important problem in Natural Language Processing. It has been explored for a number of years, and it remains partially solved. In addition to its economic and educational useful…
Automated Essay ScoringData AugmentationTransfer LearningComparison of Scoring Rationales Between Large Language Models and Human Raters
Advances in automated scoring are closely aligned with advances in machine-learning and natural-language-processing techniques. With recent progress in large language models (LLMs), the use of ChatGPT, Gemini, Claude, an…
Improving Performance of Automated Essay Scoring by using back-translation essays and adjusted scores
Automated essay scoring plays an important role in judging students' language abilities in education. Traditional approaches use handcrafted features to score and are time-consuming and complicated. Recently, neural netw…
Automated Essay ScoringFeature EngineeringH-AES: Towards Automated Essay Scoring for Hindi
The use of Natural Language Processing (NLP) for Automated Essay Scoring (AES) has been well explored in the English language, with benchmark models exhibiting performance comparable to human scorers. However, AES in Hin…
Automated Essay Scoring