A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning

2019-07-02 · WS 2019 8 · Yo Joong Choe, Jiyeon Ham, Kyubyong Park, Yeoil Yoon

Grammatical error correction can be viewed as a low-resource sequence-to-sequence task, because publicly available parallel corpora are limited. To tackle this challenge, we first generate erroneous versions of large unannotated corpora using a realistic noising function. The resulting parallel corpora are subsequently used to pre-train Transformer models. Then, by sequentially applying transfer learning, we adapt these models to the domain and style of the test set. Combined with a context-aware neural spellchecker, our system achieves competitive results in both restricted and low resource tracks in ACL 2019 BEA Shared Task. We release all of our code and materials for reproducibility.

📄 PDF Abstract BibTeX arXiv:1907.01256

Code (2)

kakaobrain/helo_word 공식 구현 pytorch

kakaobrain/helo-word pytorch

Tasks

Grammatical Error Correction Transfer Learning

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Residual Connection 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

ReLU How Do I Communicate to Expedia? How Do I Communicate to Expedia? – Call ☎️ +1-(888) 829 (0881) or +1-805-330-4056 or +1-805-330-4056 for Live Support & Special Travel…

Chinese Grammatical Error Correction Based on Hybrid Models with Data Augmentation

2020-12-01 · AACL (NLP-TEA) 2020 12 · Yi Wang, Ruibin Yuan, Yan‘gen Luo, Yufang Qin 외

A better Chinese Grammatical Error Diagnosis (CGED) system for automatic Grammatical Error Correction (GEC) can benefit foreign Chinese learners and lower Chinese learning barriers. In this paper, we introduce our soluti…

Data Augmentation Grammatical Error Correction

Combining ResNet and Transformer for Chinese Grammatical Error Diagnosis

2020-10-01 · Shaolei Wang, Baoxin Wang, Jiefu Gong, Zhongyuan Wang 외

Grammatical error diagnosis is an important task in natural language processing. This paper introduces our system at NLPTEA-2020 Task: Chinese Grammatical Error Diagnosis (CGED). CGED aims to diagnose four types of gramm…

Position

Enhancing Grammatical Error Correction Systems with Explanations

2023-05-25 · Yuejiao Fei, Leyang Cui, Sen yang, Wai Lam 외

Grammatical error correction systems improve written communication by detecting and correcting language mistakes. To help language learners better understand why the GEC system makes a certain correction, the causes of e…

Grammatical Error Correction

Organic Data-Driven Approach for Turkish Grammatical Error Correction and LLMs

2024-05-24 · Asım Ersoy, Olcay Taner Yildiz

Grammatical Error Correction has seen significant progress with the recent advancements in deep learning. As those methods require huge amounts of data, synthetic datasets are being built to fill this gap. Unfortunately,…

Grammatical Error Correction

Efficient and Interpretable Grammatical Error Correction with Mixture of Experts

2024-10-30 · Muhammad Reza Qorib, Alham Fikri Aji, Hwee Tou Ng

Error type information has been widely used to improve the performance of grammatical error correction (GEC) models, whether for generating corrections, re-ranking them, or combining GEC models. Combining GEC models that…

Grammatical Error Correction Mixture-of-Experts Re-Ranking