When is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation
Subword segmentation algorithms have been a \textit{de facto} choice when building neural machine translation systems. However, most of them need to learn a segmentation model based on some heuristics, which may produce sub-optimal segmentation. This can be problematic in some scenarios when the target language has rich morphological changes or there is not enough data for learning compact composition rules. Translating at fully character level has the potential to alleviate the issue, but empirical performances of character-based models has not been fully explored. In this paper, we present an in-depth comparison between character-based and subword-based NMT systems under three settings: translating to typologically diverse languages, training with low resource, and adapting to unseen domains. Experiment results show strong competitiveness of character-based models. Further analyses show that compared to subword-based models, character-based models are better at handling morphological phenomena, generating rare and unknown words, and more suitable for transferring to unseen domains.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationNMTSegmentationTranslationSimilar Papers 제목 키워드 기반
An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization
In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages: English, German, Hungarian, Icelandic, and Swedish. The NMT models are at different levels, have differ…
Machine TranslationNMTTranslation"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
Whole word masking (WWM), which masks all subwords corresponding to a word at once, makes a better English BERT model. For the Chinese language, however, there is no subword because each token is an atomic character. The…
Grammatical Error CorrectionLanguage ModelingLanguage ModellingMasked Language Modeling+1"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
Whole word masking (WWM), which masks all subwords corresponding to a word at once, makes a better English BERT model. For the Chinese language, however, there is no subword because each token is an atomic character. The…
Grammatical Error CorrectionLanguage ModelingLanguage ModellingMasked Language Modeling+1“Is Whole Word Masking Always Better for Chinese BERT?”: Probing on Chinese Grammatical Error Correction
Whole word masking (WWM), which masks all subwords corresponding to a word at once, makes a better English BERT model. For the Chinese language, however, there is no subword because each token is an atomic character. The…
Grammatical Error CorrectionLanguage ModelingLanguage ModellingMasked Language Modeling+1Subword-Delimited Downsampling for Better Character-Level Translation
Subword-level models have been the dominant paradigm in NLP. However, character-level models have the benefit of seeing each character individually, providing the model with more detailed information that ultimately coul…
Machine TranslationTranslation