Stacking Neural Network Models for Automatic Short Answer Scoring
Automatic short answer scoring is one of the text classification problems to assess students' answers during exams automatically. Several challenges can arise in making an automatic short answer scoring system, one of which is the quantity and quality of the data. The data labeling process is not easy because it requires a human annotator who is an expert in their field. Further, the data imbalance process is also a challenge because the number of labels for correct answers is always much less than the wrong answers. In this paper, we propose the use of a stacking model based on neural network and XGBoost for classification process with sentence embedding feature. We also propose to use data upsampling method to handle imbalance classes and hyperparameters optimization algorithm to find a robust model automatically. We use Ukara 1.0 Challenge dataset and our best model obtained an F1-score of 0.821 exceeding the previous work at the same dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationSentenceSentence EmbeddingSentence-Embeddingtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
ETS: Domain Adaptation and Stacking for Short Answer Scoring
Implicit Phenomena in Short-answer Scoring Data
Short-answer scoring is the task of assessing the correctness of a short text given as response to a question that can come from a variety of educational scenarios. As only content, not form, is important, the exact word…
Word EmbeddingsAutomatic scoring of short answers using justification cues estimated by BERT
Automated scoring technology for short-answer questions has been attracting attention to improve the fairness of scoring and reduce the burden on the scorer. In general, a large amount of data is required to train an aut…
FairnessIs getting the right answer just about choosing the right words? The role of syntactically-informed features in short answer scoring
Developments in the educational landscape have spurred greater interest in the problem of automatically scoring short answer questions. A recent shared task on this topic revealed a fundamental divide in the modeling app…
Don't take ``nswvtnvakgxpm'' for an answer --The surprising vulnerability of automatic content scoring systems to adversarial input
Automatic content scoring systems are widely used on short answer tasks to save human effort. However, the use of these systems can invite cheating strategies, such as students writing irrelevant answers in the hopes of …