VerAs: Verify then Assess STEM Lab Reports
With an increasing focus in STEM education on critical thinking skills, science writing plays an ever more important role in curricula that stress inquiry skills. A recently published dataset of two sets of college level lab reports from an inquiry-based physics curriculum relies on analytic assessment rubrics that utilize multiple dimensions, specifying subject matter knowledge and general components of good explanations. Each analytic dimension is assessed on a 6-point scale, to provide detailed feedback to students that can help them improve their science writing skills. Manual assessment can be slow, and difficult to calibrate for consistency across all students in large classes. While much work exists on automated assessment of open-ended questions in STEM subjects, there has been far less work on long-form writing such as lab reports. We present an end-to-end neural architecture that has separate verifier and assessment modules, inspired by approaches to Open Domain Question Answering (OpenQA). VerAs first verifies whether a report contains any content relevant to a given rubric dimension, and if so, assesses the relevant sentences. On the lab reports, VerAs outperforms multiple baselines based on OpenQA systems or Automated Essay Scoring (AES). VerAs also performs well on an analytic rubric for middle school physics essays.
Code (1)
Tasks
Automated Essay ScoringOpen-Domain Question AnsweringQuestion AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
Post-hoc context erasing over the KV cache is challenging because a local edit has a global consequence: once a span has been processed, its influence propagates into the cached states of all subsequent tokens. This issu…
Clinically Accurate Chest X-Ray Report Generation
The automatic generation of radiology reports given medical radiographs has significant potential to operationally and improve clinical patient care. A number of prior works have focused on this problem, employing advanc…
Reinforcement LearningText GenerationFact-checking AI-generated news reports: Can LLMs catch their own lies?
In this paper, we evaluate the ability of Large Language Models (LLMs) to assess the veracity of claims in ''news reports'' generated by themselves or other LLMs. Our goal is to determine whether LLMs can effectively fac…
DiagnosticFact CheckingRAGRetrieval-augmented GenerationPrivEraserVerify: Efficient, Private, and Verifiable Federated Unlearning
Federated learning (FL) enables collaborative model training without sharing raw data, offering a promising path toward privacy preserving artificial intelligence. However, FL models may still memorize sensitive informat…
Federated LearningEmulating Clinical Quality Muscle B-mode Ultrasound Images from Plane Wave Images Using a Two-Stage Machine Learning Model
Research ultrasound scanners such as the Verasonics Vantage often lack the advanced image processing algorithms used by clinical systems. Image quality is even lower in plane wave imaging - often used for shear wave elas…