Analyzing Large Language Models for Classroom Discussion Assessment
Automatically assessing classroom discussion quality is becoming increasingly feasible with the help of new NLP advancements such as large language models (LLMs). In this work, we examine how the assessment performance of 2 LLMs interacts with 3 factors that may affect performance: task formulation, context length, and few-shot examples. We also explore the computational efficiency and predictive consistency of the 2 LLMs. Our results suggest that the 3 aforementioned factors do affect the performance of the tested LLMs and there is a relation between consistency and performance. We recommend a LLM-based assessment approach that has a good balance in terms of predictive performance, computational efficiency, and consistency.
Code (1)
Tasks
Computational EfficiencySimilar Papers 제목 키워드 기반
Utilizing Natural Language Processing for Automated Assessment of Classroom Discussion
Rigorous and interactive class discussions that support students to engage in high-level thinking and reasoning are essential to learning and are a central component of most teaching interventions. However, formally asse…
Annotating Student Talk in Text-based Classroom Discussions
Classroom discussions in English Language Arts have a positive effect on students' reading, writing and reasoning skills. Although prior work has largely focused on teacher talk and student-teacher interactions, we focus…
SpecificityPredicting Specificity in Classroom Discussion
High quality classroom discussion is important to student development, enhancing abilities to express claims, reason about other students' claims, and retain information for longer periods of time. Previous small-scale s…
SpecificityDiscussion Tracker: Supporting Teacher Learning about Students' Collaborative Argumentation in High School Classrooms
Teaching collaborative argumentation is an advanced skill that many K-12 teachers struggle to develop. To address this, we have developed Discussion Tracker, a classroom discussion analytics system based on novel algorit…
SpecificityWhat Would a Teacher Do? Predicting Future Talk Moves
Recent advances in natural language processing (NLP) have the ability to transform how classroom learning takes place. Combined with the increasing integration of technology in today's classrooms, NLP systems leveraging …
Question Answering