Powergrading: a Clustering Approach to Amplify Human Effort for Short Answer Grading
We introduce a new approach to the machine-assisted grading of short answer questions. We follow past work in automated grading by first training a similarity metric between student responses, but then go on to use this metric to group responses into clusters and subclusters. The resulting groupings allow teachers to grade multiple responses with a single action, provide rich feedback to groups of similar answers, and discover modalities of misunderstanding among students; we refer to this amplification of grader effort as {`}powergrading.{''} We develop the means to further reduce teacher effort by automatically performing actions when an answer key is available. We show results in terms of grading progress with a small {`}budget{''} of human actions, both from our method and an LDA-based approach, on a test corpus of 10 questions answered by 698 respondents.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringSimilar Papers 제목 키워드 기반
LUMI: Unsupervised Intent Clustering with Multiple Pseudo-Labels
In this paper, we propose an intuitive, training-free and label-free method for intent clustering in conversational search. Current approaches to short text clustering use LLM-generated pseudo-labels to enrich text repre…
Short Text ClusteringClustering of Multi-Word Named Entity variants: Multilingual Evaluation
Multi-word entities, such as organisation names, are frequently written in many different ways. We have previously automatically identified over one million acronym pairs in 22 languages, consisting of their short form (…
ClusteringDocument ClassificationFormQuestion AnsweringImproving Neural Networks by Adopting Amplifying and Attenuating Neurons
In the present study, an amplifying neuron and attenuating neuron, which can be easily implemented into neural networks without any significant additional computational effort, are proposed. The activated output value is…
Human-interpretable clustering of short-text using large language models
Clustering short text is a difficult problem, due to the low word co-occurrence between short text documents. This work shows that large language models (LLMs) can overcome the limitations of traditional clustering appro…
ClusteringShort Text ClusteringText ClusteringPost Hoc Explanations of Language Models Can Improve Language Models
Large Language Models (LLMs) have demonstrated remarkable capabilities in performing complex tasks. Moreover, recent research has shown that incorporating human-annotated rationales (e.g., Chain-of-Thought prompting) dur…
In-Context Learning