Misspelling Correction with Pre-trained Contextual Language Model
Spelling irregularities, known now as spelling mistakes, have been found for several centuries. As humans, we are able to understand most of the misspelled words based on their location in the sentence, perceived pronunciation, and context. Unlike humans, computer systems do not possess the convenient auto complete functionality of which human brains are capable. While many programs provide spelling correction functionality, many systems do not take context into account. Moreover, Artificial Intelligence systems function in the way they are trained on. With many current Natural Language Processing (NLP) systems trained on grammatically correct text data, many are vulnerable against adversarial examples, yet correctly spelled text processing is crucial for learning. In this paper, we investigate how spelling errors can be corrected in context, with a pre-trained language model BERT. We present two experiments, based on BERT and the edit distance algorithm, for ranking and selecting candidate corrections. The results of our experiments demonstrated that when combined properly, contextual word embeddings of BERT and edit distance are capable of effectively correcting spelling errors.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingSentenceSpelling CorrectionWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
NeuSpell: A Neural Spelling Correction Toolkit
We introduce NeuSpell, an open-source toolkit for spelling correction in English. Our toolkit comprises ten different models, and benchmarks them on naturally occurring misspellings from multiple sources. We find that ma…
Spelling CorrectionMaps Search Misspelling Detection Leveraging Domain-Augmented Contextual Representations
Building an independent misspelling detector and serve it before correction can bring multiple benefits to speller and other search components, which is particularly true for the most commonly deployed noisy-channel base…
Representation LearningContext-aware Stand-alone Neural Spelling Correction
Existing natural language processing systems are vulnerable to noisy inputs resulting from misspellings. On the contrary, humans can easily infer the corresponding correct words from their misspellings and surrounding co…
Language ModelingLanguage ModellingSpelling CorrectionRetrieval Augmented Spelling Correction for E-Commerce Applications
The rapid introduction of new brand names into everyday language poses a unique challenge for e-commerce spelling correction services, which must distinguish genuine misspellings from novel brand names that use unconvent…
Language ModelingLanguage ModellingLarge Language ModelRAG+3Parallel Spell-Checking Algorithm Based on Yahoo! N-Grams Dataset
Spell-checking is the process of detecting and sometimes providing suggestions for incorrectly spelled words in a text. Basically, the larger the dictionary of a spell-checker is, the higher is the error detection rate; …
Articles