Taking the Correction Difficulty into Account in Grammatical Error Correction Evaluation
This paper presents performance measures for grammatical error correction which take into account the difficulty of error correction. To the best of our knowledge, no conventional measure has such functionality despite the fact that some errors are easy to correct and others are not. The main purpose of this work is to provide a way of determining the difficulty of error correction and to motivate researchers in the domain to attack such difficult errors. The performance measures are based on the simple idea that the more systems successfully correct an error, the easier it is considered to be. This paper presents a set of algorithms to implement this idea. It evaluates the performance measures quantitatively and qualitatively on a wide variety of corpora and systems, revealing that they agree with our intuition of correction difficulty. A scorer and difficulty weight data based on the algorithms have been made available on the web.
Code (1)
Tasks
Grammatical Error CorrectionSimilar Papers 제목 키워드 기반
Collecting fluency corrections for spoken learner English
We present crowdsourced collection of error annotations for transcriptions of spoken learner English. Our emphasis in data collection is on fluency corrections, a more complete correction than has traditionally been aime…
Grammatical Error CorrectionGrammatical Error DetectionMachine TranslationExploring Human-judged and Automatically-induced Correction Difficulty for Grammatical Error Correction
While grammatical error correction (GEC) has improved in its correction performance, one of the key challenges in GEC research still remains in evaluation. Specifically, all errors are equally treated in the conventional…
Grammatical Error CorrectionArbESC+: Arabic Enhanced Edit Selection System Combination for Grammatical Error Correction Resolving conflict and improving system combination in Arabic GEC
Grammatical Error Correction (GEC) is an important aspect of natural language processing. Arabic has a complicated morphological and syntactic structure, posing a greater challenge than other languages. Even though moder…
Grammatical Error CorrectionLoss-Aware Curriculum Learning for Chinese Grammatical Error Correction
Chinese grammatical error correction (CGEC) aims to detect and correct errors in the input Chinese sentences. Recently, Pre-trained Language Models (PLMS) have been employed to improve the performance. However, current a…
Grammatical Error CorrectionAutomatic Classification of Russian Learner Errors
Grammatical Error Correction systems are typically evaluated overall, without taking into consideration performance on individual error types because system output is not annotated with respect to error type. We introduc…
ClassificationGrammatical Error Correction