ACE: Automatic Colloquialism, Typographical and Orthographic Errors Detection for Chinese Language
We present a system called ACE for Automatic Colloquialism and Errors detection for written Chinese. ACE is based on the combination of N-gram model and rule-base model. Although it focuses on detecting colloquial Cantonese (a dialect of Chinese) at the current stage, it can be extended to detect other dialects. We chose Cantonese becauase it has many interesting properties, such as unique grammar system and huge colloquial terms, that turn the detection task extremely challenging. We conducted experiments using real data and synthetic data. The results indicated that ACE is highly reliable and effective.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Persian Typographical Error Type Detection Using Deep Neural Networks on Algorithmically-Generated Misspellings
Spelling correction is a remarkable challenge in the field of natural language processing. The objective of spelling correction tasks is to recognize and rectify spelling errors automatically. The development of applicat…
Spelling Correctiontoken-classificationToken ClassificationCorrecting the Autocorrect: Context-Aware Typographical Error Correction via Training Data Augmentation
In this paper, we explore the artificial generation of typographical errors based on real-world statistics. We first draw on a small set of annotated data to compute spelling error statistics. These are then invoked to i…
BIG-bench Machine LearningData AugmentationCorrecting Arabic Soft Spelling Mistakes using BiLSTM-based Machine Learning
Soft spelling errors are a class of spelling mistakes that is widespread among native Arabic speakers and foreign learners alike. Some of these errors are typographical in nature. They occur due to orthographic variation…
BIG-bench Machine LearningDecoderSpelling CorrectionE-Bench: Towards Evaluating the Ease-of-Use of Large Language Models
Most large language models (LLMs) are sensitive to prompts, and another synonymous expression or a typo may lead to unexpected results for the model. Composing an optimal prompt for a specific demand lacks theoretical su…
GM-RKB WikiText Error Correction Task and Baselines
We introduce the GM-RKB WikiText Error Correction Task for the automatic detection and correction of typographical errors in WikiText annotated pages. The included corpus is based on a snapshot of the GM-RKB domain-speci…
Language ModelingLanguage Modelling