Neural Predictive Text for Grammatical Error Prevention
In this paper we study the potential of two neural language models, an LSTM and an autoregressive language model GPT-2, to predict possible correction tokens in erroneous sentences and to predict the next token in randomly sliced correct sentences, in the aim of establishing a new Grammatical Error Correction (GEC) subarea, for which we coin the term Grammatical Error Prevention (GEP). Systems that could assist in GEP, such as language models, are expected to predict elements and therefore prevent grammatical errors in advance. Our findings show that GPT-2 can predict 29% of the correct tokens with one prediction. Accuracy rises up to 44% when the top 3 predictions are considered. To test the pedagogical capacity of such a model, we also experimented with real English as a second language (ESL) learners. By equipping GPT-2 to generate text that functions as potential continuation of the learners' sentences, we created a small corpus of the learners' writings and analyzed their errors along with their frequencies.
Code (0)
등록된 구현이 없습니다.
Tasks
Grammatical Error CorrectionLanguage ModelingLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SMT Errors Requiring Grammatical Knowledge for Prevention
Prevention is better than cure: a case study of the abnormalities detection in the chest
Prevention is better than cure. This old truth applies not only to the prevention of diseases but also to the prevention of issues with AI models used in medicine. The source of malfunctioning of predictive models often …
FairnessGrammatical Error Detection Using Error- and Grammaticality-Specific Word Embeddings
In this study, we improve grammatical error detection by learning word embeddings that consider grammaticality and error patterns. Most existing algorithms for learning word embeddings usually model only the syntactic co…
Grammatical Error DetectionLearning Word EmbeddingsWord EmbeddingsContrastive Analysis with Predictive Power: Typology Driven Estimation of Grammatical Error Distributions in ESL
This work examines the impact of cross-linguistic transfer on grammatical errors in English as Second Language (ESL) texts. Using a computational framework that formalizes the theory of Contrastive Analysis (CA), we demo…
Language AcquisitionAutomatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study
Sequence-to-sequence (seq2seq) models have achieved tremendous success in text generation tasks. However, there is no guarantee that they can always generate sentences without grammatical errors. In this paper, we presen…
Formality Style TransferGrammatical Error CorrectionMachine TranslationSentence+4