paper-with-me

Papers

Automatic rubric-based content grading for clinical notes

2019-11-01 · WS 2019 11 · Wen-wai Yim, Ashley Mills, Harold Chun, Teresa Hashiguchi, Justin Yew, Bryan Lu

Clinical notes provide important documentation critical to medical care, as well as billing and legal needs. Too little information degrades quality of care; too much information impedes care. Training for clinical note documentation is highly variable, depending on institutions and programs. In this work, we introduce the problem of automatic evaluation of note creation through rubric-based content grading, which has the potential for accelerating and regularizing clinical note documentation training. To this end, we describe our corpus creation methods as well as provide simple feature-based and neural network baseline systems. We further provide tagset and scaling experiments to inform readers of plausible expected performances. Our baselines show promising results with content point accuracy and kappa values at 0.86 and 0.71 on the test set.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Test 설명 없음

Similar Papers 제목 키워드 기반

Unveiling Scoring Processes: Dissecting the Differences between LLMs and Human Graders in Automatic Scoring

2024-07-04 · Xuansheng Wu, Padmaja Pravin Saraf, Gyeonggeon Lee, Ehsan Latif 외

Large language models (LLMs) have demonstrated strong potential in performing automatic scoring for constructed response assessments. While constructed responses graded by humans are usually based on given grading rubric…

Logical Reasoning

The USMLE® Step 2 Clinical Skills Patient Note Corpus

2022-07-01 · NAACL 2022 7 · Victoria Yaneva, Janet Mee, Le Ha, Polina Harik 외

This paper presents a corpus of 43,985 clinical patient notes (PNs) written by 35,156 examinees during the high-stakes USMLE® Step 2 Clinical Skills examination. In this exam, examinees interact with standardized patient…

Ratas framework: A comprehensive genai-based approach to rubric-based marking of real-world textual exams

2025-05-27 · Masoud Safilian, Amin Beheshti, Stephen Elbourn

Automated answer grading is a critical challenge in educational technology, with the potential to streamline assessment processes, ensure grading consistency, and provide timely feedback to students. However, existing ap…

Automated Grading of Handwritten Mathematics Using Vision-Capable LLMs

2026-05-18 · Jacob Levine, Miguel Aenlle, Craig Zilles, Matthew West 외 arxiv

Automated grading systems have enabled scalable assessment for many response types, but handwritten mathematics remains a barrier due to the complexity of multi-step solutions. Vision-capable large language models (LLMs)…

LLM-based Automated Grading with Human-in-the-Loop

2025-04-07 · Hang Li, Yucheng Chu, Kaiqi Yang, Yasemin Copur-Gencturk 외

The rise of artificial intelligence (AI) technologies, particularly large language models (LLMs), has brought significant advancements to the field of education. Among various applications, automatic short answer grading…

automatic short answer grading