Enabling Robust Grammatical Error Correction in New Domains: Data Sets, Metrics, and Analyses
Until now, grammatical error correction (GEC) has been primarily evaluated on text written by non-native English speakers, with a focus on student essays. This paper enables GEC development on text written by native speakers by providing a new data set and metric. We present a multiple-reference test corpus for GEC that includes 4,000 sentences in two new domains (formal and informal writing by native English speakers) and 2,000 sentences from a diverse set of non-native student writing. We also collect human judgments of several GEC systems on this new test set and perform a meta-evaluation, assessing how reliable automatic metrics are across these domains. We find that commonly used GEC metrics have inconsistent performance across domains, and therefore we propose a new ensemble metric that is robust on all three domains of text.
Code (0)
등록된 구현이 없습니다.
Tasks
Grammatical Error CorrectionSimilar Papers 제목 키워드 기반
CL$^2$GEC: A Multi-Discipline Benchmark for Continual Learning in Chinese Literature Grammatical Error Correction
The growing demand for automated writing assistance in diverse academic domains highlights the need for robust Chinese Grammatical Error Correction (CGEC) systems that can adapt across disciplines. However, existing CGEC…
Grammatical Error CorrectionContinual LearningGrammatical Error Correction with Contrastive Learning in Low Error Density Domains
Although grammatical error correction (GEC) has achieved good performance on texts written by learners of English as a second language, performance on low error density domains where texts are written by English speakers…
Contrastive LearningGrammatical Error CorrectionSentenceGrammatical Error Correction in Low Error Density Domains: A New Benchmark and Analyses
Evaluation of grammatical error correction (GEC) systems has primarily focused on essays written by non-native learners of English, which however is only part of the full spectrum of GEC applications. We aim to broaden t…
Grammatical Error CorrectionLanguage ModelingLanguage ModellingIMPARA: Impact-Based Metric for GEC Using Parallel Data
Automatic evaluation of grammatical error correction (GEC) is essential in developing useful GEC systems. Existing methods for automatic evaluation require multiple reference sentences or manual scores. However, such res…
Grammatical Error CorrectionN-gram Model for Chinese Grammatical Error Diagnosis
Detection and correction of Chinese grammatical errors have been two of major challenges for Chinese automatic grammatical error diagnosis.This paper presents an N-gram model for automatic detection and correction of Chi…
Language ModelingLanguage Modellingmodel