MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction
This paper presents MuCGEC, a multi-reference multi-source evaluation dataset for Chinese Grammatical Error Correction (CGEC), % based on newly proposed annotation guidelines, consisting of 7,063 sentences from three different Chinese-as-a-Second-Language (CSL) learner sources. Each sentence has been corrected by three annotators, and their corrections are meticulously reviewed by an expert, resulting in 2.3 references on average per sentence. We conduct experiments with two mainstream CGEC models, i.e., the sequence-to-sequence (Seq2Seq) model and the sequence-to-edit (Seq2Edit) model, both enhanced with large pretrained language models, achieving competitive benchmark performance on previous and our datasets. We also discuss the CGEC evaluation methodologies, including the effect of multiple references and using a char-based metric. We will release our annotation guidelines, data, and code.
Code (0)
등록된 구현이 없습니다.
Tasks
Grammatical Error CorrectionSentenceSimilar Papers 제목 키워드 기반
MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction
This paper presents MuCGEC, a multi-reference multi-source evaluation dataset for Chinese Grammatical Error Correction (CGEC), consisting of 7,063 sentences collected from three Chinese-as-a-Second-Language (CSL) learner…
Grammatical Error CorrectionSentenceChinese Word Boundary Recovery through Character Alignment Projection
Chinese word segmentation is especially fragile in non-standard text, where language learner errors and other character-level divergences disrupt the word boundaries assumed by downstream annotation and evaluation. This …
Chinese Word SegmentationCLEME: Debiasing Multi-reference Evaluation for Grammatical Error Correction
Evaluating the performance of Grammatical Error Correction (GEC) systems is a challenging task due to its subjectivity. Designing an evaluation metric that is as objective as possible is crucial to the development of GEC…
Grammatical Error CorrectionMulti-Dimensional Machine Translation Evaluation: Model Evaluation and Resource for Korean
Almost all frameworks for the manual or automatic evaluation of machine translation characterize the quality of an MT output with a single number. An exception is the Multidimensional Quality Metrics (MQM) framework whic…
Machine TranslationSentenceTranslationOmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation
Reference-to-video (R2V) generation is evolving toward increasingly general and versatile reference control, giving rise to the emerging paradigm of omni R2V generation. However, existing benchmarks fall short of these e…
Video Generation