A Combination of BERT and Transformer for Vietnamese Spelling Correction
Recently, many studies have shown the efficiency of using Bidirectional Encoder Representations from Transformers (BERT) in various Natural Language Processing (NLP) tasks. Specifically, English spelling correction task that uses Encoder-Decoder architecture and takes advantage of BERT has achieved state-of-the-art result. However, to our knowledge, there is no implementation in Vietnamese yet. Therefore, in this study, a combination of Transformer architecture (state-of-the-art for Encoder-Decoder model) and BERT was proposed to deal with Vietnamese spelling correction. The experiment results have shown that our model outperforms other approaches as well as the Google Docs Spell Checking tool, achieves an 86.24 BLEU score on this task.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderSpelling CorrectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Hierarchical Transformer Encoders for Vietnamese Spelling Correction
In this paper, we propose a Hierarchical Transformer model for Vietnamese spelling correction problem. The model consists of multiple Transformer encoders and utilizes both character-level and word-level to detect errors…
Spelling CorrectionVSEC: Transformer-based Model for Vietnamese Spelling Correction
Spelling error correction is one of topics which have a long history in natural language processing. Although previous studies have achieved remarkable results, challenges still exist. In the Vietnamese language, a state…
modelSpelling CorrectionHierarchical Character Tagger for Short Text Spelling Error Correction
State-of-the-art approaches to spelling error correction problem include Transformer-based Seq2Seq models, which require large training sets and suffer from slow inference time; and sequence labeling models based on Tran…
Language ModelingLanguage ModellingVartani Spellcheck -- Automatic Context-Sensitive Spelling Correction of OCR-generated Hindi Text Using BERT and Levenshtein Distance
Traditional Optical Character Recognition (OCR) systems that generate text of highly inflectional Indic languages like Hindi tend to suffer from poor accuracy due to a wide alphabet set, compound characters and difficult…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+3A Comprehensive Approach to Misspelling Correction with BERT and Levenshtein Distance
Writing, as an omnipresent form of human communication, permeates nearly every aspect of contemporary life. Consequently, inaccuracies or errors in written communication can lead to profound consequences, ranging from fi…
Language ModelingLanguage Modelling