Chinese Spelling Error Detection Using a Fusion Lattice LSTM
Spelling error detection serves as a crucial preprocessing in many natural language processing applications. Due to the characteristics of Chinese Language, Chinese spelling error detection is more challenging than error detection in English. Existing methods are mainly under a pipeline framework, which artificially divides error detection process into two steps. Thus, these methods bring error propagation and cannot always work well due to the complexity of the language environment. Besides existing methods only adopt character or word information, and ignore the positive effect of fusing character, word, pinyin1 information together. We propose an LF-LSTM-CRF model, which is an extension of the LSTMCRF with word lattices and character-pinyin-fusion inputs. Our model takes advantage of the end-to-end framework to detect errors as a whole process, and dynamically integrates character, word and pinyin information. Experiments on the SIGHAN data show that our LF-LSTM-CRF outperforms existing methods with similar external resources consistently, and confirm the feasibility of adopting the end-to-end framework and the availability of integrating of character, word and pinyin information.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
An Error-Guided Correction Model for Chinese Spelling Error Correction
Although existing neural network approaches have achieved great success on Chinese spelling correction, there is still room to improve. The model is required to avoid over-correction and to distinguish a correct token fr…
Chinese Spelling Error CorrectionSpelling CorrectionuChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers
The task of Chinese Spelling Check (CSC) is aiming to detect and correct spelling errors that can be found in the text. While manually annotating a high-quality dataset is expensive and time-consuming, thus the scale of …
Language ModelingLanguage ModellingSentenceA Coin Has Two Sides: A Novel Detector-Corrector Framework for Chinese Spelling Correction
Chinese Spelling Correction (CSC) stands as a foundational Natural Language Processing (NLP) task, which primarily focuses on the correction of erroneous characters in Chinese texts. Certain existing methodologies opt to…
Spelling CorrectionSpellBERT: A Lightweight Pretrained Model for Chinese Spelling Check
Chinese Spelling Check (CSC) is to detect and correct Chinese spelling errors. Many models utilize a predefined confusion set to learn a mapping between correct characters and its visually similar or phonetically similar…
Graph Neural NetworkLanguage ModelingLanguage ModellingOptical Character Recognition (OCR)MCSCSet: A Specialist-annotated Dataset for Medical-domain Chinese Spelling Correction
Chinese Spelling Correction (CSC) is gaining increasing attention due to its promise of automatically detecting and correcting spelling errors in Chinese texts. Despite its extensive use in many applications, like search…
Optical Character RecognitionOptical Character Recognition (OCR)Spelling Correction