Utilizing Natural Language Processing for Automated Assessment of Classroom Discussion
Rigorous and interactive class discussions that support students to engage in high-level thinking and reasoning are essential to learning and are a central component of most teaching interventions. However, formally assessing discussion quality 'at scale' is expensive and infeasible for most researchers. In this work, we experimented with various modern natural language processing (NLP) techniques to automatically generate rubric scores for individual dimensions of classroom text discussion quality. Specifically, we worked on a dataset of 90 classroom discussion transcripts consisting of over 18000 turns annotated with fine-grained Analyzing Teaching Moves (ATM) codes and focused on four Instructional Quality Assessment (IQA) rubrics. Despite the limited amount of data, our work shows encouraging results in some of the rubrics while suggesting that there is room for improvement in the others. We also found that certain NLP approaches work better for certain rubrics.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Using Item Response Theory to Measure Gender and Racial Bias of a BERT-based Automated English Speech Assessment System
Recent advances in natural language processing and transformer-based models have made it easier to implement accurate, automated English speech assessments. Yet, without careful examination, applications of these models …
NLP and Education: using semantic similarity to evaluate filled gaps in a large-scale Cloze test in the classroom
This study examines the applicability of the Cloze test, a widely used tool for assessing text comprehension proficiency, while highlighting its challenges in large-scale implementation. To address these limitations, an …
Cloze TestReading ComprehensionSemantic SimilaritySemantic Textual Similarity+1Translation Quality Assessment: A Brief Survey on Manual and Automatic Methods
To facilitate effective translation modeling and translation studies, one of the crucial questions to address is how to assess translation quality. From the perspectives of accuracy, reliability, repeatability and cost, …
Machine TranslationNatural Language UnderstandingText GenerationText Summarization+1Neural Automated Writing Evaluation with Corrective Feedback
The utilization of technology in second language learning and teaching has become ubiquitous. For the assessment of writing specifically, automated writing evaluation (AWE) and grammatical error correction (GEC) have bec…
Automated Writing EvaluationGrammatical Error CorrectionTeamwork Dimensions Classification Using BERT
Teamwork is a necessary competency for students that is often inadequately assessed. Towards providing a formative assessment of student teamwork, an automated natural language processing approach was developed to identi…
ClassificationLanguage ModelingLanguage ModellingTransfer Learning