Proficiency Matters Quality Estimation in Grammatical Error Correction
This study investigates how supervised quality estimation (QE) models of grammatical error correction (GEC) are affected by the learners' proficiency with the data. QE models for GEC evaluations in prior work have obtained a high correlation with manual evaluations. However, when functioning in a real-world context, the data used for the reported results have limitations because prior works were biased toward data by learners with relatively high proficiency levels. To address this issue, we created a QE dataset that includes multiple proficiency levels and explored the necessity of performing proficiency-wise evaluation for QE of GEC. Our experiments demonstrated that differences in evaluation dataset proficiency affect the performance of QE models, and proficiency-wise evaluation helps create more robust models.
Code (0)
등록된 구현이 없습니다.
Tasks
Grammatical Error CorrectionSimilar Papers 제목 키워드 기반
ProQE: Proficiency-wise Quality Estimation dataset for Grammatical Error Correction
This study investigates how supervised quality estimation (QE) models of grammatical error correction (GEC) are affected by the learners’ proficiency with the data. QE models for GEC evaluations in prior work have obtain…
Grammatical Error CorrectionExploiting the English Grammar Profile for L2 grammatical analysis with LLMs
Evaluating the grammatical competence of second language (L2) learners is essential both for providing targeted feedback and for assessing proficiency. To achieve this, we propose a novel framework leveraging the English…
Grammatical Error CorrectionCross-corpora experiments of automatic proficiency assessment and error detection for spoken English
The growing demand for learning English as a second language has led to an increasing interest in automatic approaches for assessing spoken language proficiency. One of the most significant challenges in this field is th…
JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction
We present a new parallel corpus, JHU FLuency-Extended GUG corpus (JFLEG) for developing and evaluating grammatical error correction (GEC). Unlike other corpora, it represents a broad range of language proficiency levels…
Grammatical Error CorrectionEvaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiency
The writing examples of English language learners may be different from those of native speakers. Given that there is a significant differences in second language (L2) learners' error types by their proficiency levels, t…
Grammatical Error Correction