Efficient and Interpretable Grammatical Error Correction with Mixture of Experts
Error type information has been widely used to improve the performance of grammatical error correction (GEC) models, whether for generating corrections, re-ranking them, or combining GEC models. Combining GEC models that have complementary strengths in correcting different error types is very effective in producing better corrections. However, system combination incurs a high computational cost due to the need to run inference on the base systems before running the combination method itself. Therefore, it would be more efficient to have a single model with multiple sub-networks that specialize in correcting different error types. In this paper, we propose a mixture-of-experts model, MoECE, for grammatical error correction. Our model successfully achieves the performance of T5-XL with three times fewer effective parameters. Additionally, our model produces interpretable corrections by also identifying the error type during inference.
Code (1)
Tasks
Grammatical Error CorrectionMixture-of-ExpertsRe-RankingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Tibyan Corpus: Balanced and Comprehensive Error Coverage Corpus Using ChatGPT for Arabic Grammatical Error Correction
Natural language processing (NLP) utilizes text data augmentation to overcome sample size constraints. Increasing the sample size is a natural and widely used strategy for alleviating these challenges. In this study, we …
Data AugmentationGrammatical Error CorrectionAutomatic Annotation and Evaluation of Error Types for Grammatical Error Correction
Until now, error type performance for Grammatical Error Correction (GEC) systems could only be measured in terms of recall because system output is not annotated. To overcome this problem, we introduce ERRANT, a grammati…
Annotated Code SearchGrammatical Error CorrectionNews AnnotationTable annotation+1CLEME2.0: Towards More Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction
The paper focuses on improving the interpretability of Grammatical Error Correction (GEC) metrics, which receives little attention in previous studies. To bridge the gap, we propose CLEME2.0, a reference-based evaluation…
Grammatical Error CorrectionExploring Human-judged and Automatically-induced Correction Difficulty for Grammatical Error Correction
While grammatical error correction (GEC) has improved in its correction performance, one of the key challenges in GEC research still remains in evaluation. Specifically, all errors are equally treated in the conventional…
Grammatical Error CorrectionEncode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction
Grammatical Error Correction (GEC) involves detecting and correcting the wrong usage of grammar. While large language models (LLMs) with in-context learning (ICL) capabilities have shown significant progress on various n…
Grammatical Error CorrectionSemantic Similarity