Rare but Severe Neural Machine Translation Errors Induced by Minimal Deletion: An Empirical Study on Chinese and English
We examine the inducement of rare but severe errors in English-Chinese and Chinese-English in-domain neural machine translation by minimal deletion of the source text with character-based models. By deleting a single character, we can induce severe translation errors. We categorize these errors and compare the results of deleting single characters and single words. We also examine the effect of training data size on the number and types of pathological cases induced by these minimal perturbations, finding significant variation. We find that deleting a word hurts overall translation score more than deleting a character, but certain errors are more likely to occur when deleting characters, with language direction also influencing the effect.
Code (1)
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
Rare but Severe Errors Induced by Minimal Deletions in English-Chinese Neural Machine Translation
We examine the inducement of rare but severe errors in English-Chinese and Chinese-English Transformer-based neural machine translation by minimal deletion in the source text. We also examine the effect of training data …
HallucinationMachine TranslationTranslationOvercoming the Rare Word Problem for Low-Resource Language Pairs in Neural Machine Translation
Among the six challenges of neural machine translation (NMT) coined by (Koehn and Knowles, 2017), rare-word problem is considered the most severe one, especially in translation of low-resource languages. In this paper, w…
Machine TranslationNMTTranslationOn the Word Alignment from Neural Machine Translation
Prior researches suggest that neural machine translation (NMT) captures word alignment through its attention mechanism, however, this paper finds attention may almost fail to capture word alignment for some NMT models. T…
Machine TranslationNMTTranslationWord AlignmentIs “moby dick” a Whale or a Bird? Named Entities and Terminology in Speech Translation
Automatic translation systems are known to struggle with rare words. Among these, named entities (NEs) and domain-specific terms are crucial, since errors in their translation can lead to severe meaning distortions. Desp…
TranslationSALTED: A Framework for SAlient Long-Tail Translation Error Detection
Traditional machine translation (MT) metrics provide an average measure of translation quality that is insensitive to the long tail of behavioral problems in MT. Examples include translation of numbers, physical units, d…
Machine TranslationNMTSentenceTranslation