A Benchmark Corpus of English Misspellings and a Minimally-supervised Model for Spelling Correction
Spelling correction has attracted a lot of attention in the NLP community. However, models have been usually evaluated on artificiallycreated or proprietary corpora. A publiclyavailable corpus of authentic misspellings, annotated in context, is still lacking. To address this, we present and release an annotated data set of 6,121 spelling errors in context, based on a corpus of essays written by English language learners. We also develop a minimallysupervised context-aware approach to spelling correction. It achieves strong results on our data: 88.12{\%} accuracy. This approach can also train with a minimal amount of annotated data (performance reduced by less than 1{\%}). Furthermore, this approach allows easy portability to new domains. We evaluate our model on data from a medical domain and demonstrate that it rivals the performance of a model trained and tuned on in-domain data.
Code (1)
Tasks
Spelling CorrectionSimilar Papers 제목 키워드 기반
Spelling Correction for Russian: A Comparative Study of Datasets and Methods
We develop a minimally-supervised model for spelling correction and evaluate its performance on three datasets annotated for spelling errors in Russian. The first corpus is a dataset of Russian social media data that was…
Cross-corpusMachine TranslationRe-RankingSpelling Correction+1ColLex.en: Automatically Generating and Evaluating a Full-form Lexicon for English
The paper describes a procedure for the automatic generation of a large full-form lexicon of English. We put emphasis on two statistical methods to lexicon extension and adjustment: in terms of a letter-based HMM and in …
FormText CategorizationText ClassificationDo We Need Neural Models to Explain Human Judgments of Acceptability?
Native speakers can judge whether a sentence is an acceptable instance of their language. Acceptability provides a means of evaluating whether computational language models are processing language in a human-like manner.…
SentenceWord EmbeddingsWord SimilarityNeuSpell: A Neural Spelling Correction Toolkit
We introduce NeuSpell, an open-source toolkit for spelling correction in English. Our toolkit comprises ten different models, and benchmarks them on naturally occurring misspellings from multiple sources. We find that ma…
Spelling CorrectionDeveloping a Twi (Asante) Dictionary from Akan Interlinear Glossed Texts
Traditionally, a lexicographer identifies the lexical items to be added to a dictionary. Here we present a corpus-based approach to dictionary compilation and describe a procedure that derives a Twi dictionary from a Typ…