Noisy Channel for Low Resource Grammatical Error Correction

2019-08-01 · WS 2019 8 · Simon Flachs, Oph{\'e}lie Lacroix, Anders S{\o}gaard

This paper describes our contribution to the low-resource track of the BEA 2019 shared task on Grammatical Error Correction (GEC). Our approach to GEC builds on the theory of the noisy channel by combining a channel model and language model. We generate confusion sets from the Wikipedia edit history and use the frequencies of edits to estimate the channel model. Additionally, we use two pre-trained language models: 1) Google{'}s BERT model, which we fine-tune for specific error types and 2) OpenAI{'}s GPT-2 model, utilizing that it can operate with previous sentences as context. Furthermore, we search for the optimal combinations of corrections using beam search.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Grammatical Error CorrectionLanguage ModelingLanguage Modelling

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Residual Connection 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Linear Warmup With Cosine Annealing Linear Warmup With Cosine Annealing is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule…

Discriminative Fine-Tuning Discriminative Fine-Tuning is a fine-tuning strategy that is used for ULMFiT type models. Instead of using the same learning rate…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Data Strategies for Low-Resource Grammatical Error Correction

2021-04-01 · EACL (BEA) 2021 4 · Simon Flachs, Felix Stahlberg, Shankar Kumar

Grammatical Error Correction (GEC) is a task that has been extensively investigated for the English language. However, for low-resource languages the best practices for training GEC systems have not yet been systematical…

Grammatical Error Correction

A Noisy Channel Model Framework for Grammatical Correction

2013-08-01 · WS 2013 8 · L. Amber Wilcox-O{'}Hearn

model Spelling Correction

Minimally-Augmented Grammatical Error Correction

2019-11-01 · WS 2019 11 · Roman Grundkiewicz, Marcin Junczys-Dowmunt

There has been an increased interest in low-resource approaches to automatic grammatical error correction. We introduce Minimally-Augmented Grammatical Error Correction (MAGEC) that does not require any error-labelled da…

Grammatical Error Correction

Grammatical Error Correction in Low-Resource Scenarios

2019-10-01 · WS 2019 11 · Jakub Náplava, Milan Straka

Grammatical error correction in English is a long studied problem with many existing systems and datasets. However, there has been only a limited research on error correction of other languages. In this paper, we present…

Grammatical Error Correction Machine Translation Translation

A Comparative Study of Synthetic Data Generation Methods for Grammatical Error Correction

2020-07-01 · WS 2020 7 · Max White, Alla Rozovskaya

Grammatical Error Correction (GEC) is concerned with correcting grammatical errors in written text. Current GEC systems, namely those leveraging statistical and neural machine translation, require large quantities of ann…

Grammatical Error Correction Machine Translation Synthetic Data Generation Translation