BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages
Morphologically-rich polysynthetic languages present a challenge for NLP systems due to data sparsity, and a common strategy to handle this issue is to apply subword segmentation. We investigate a wide variety of supervised and unsupervised morphological segmentation methods for four polysynthetic languages: Nahuatl, Raramuri, Shipibo-Konibo, and Wixarika. Then, we compare the morphologically inspired segmentation methods against Byte-Pair Encodings (BPEs) as inputs for machine translation (MT) when translating to and from Spanish. We show that for all language pairs except for Nahuatl, an unsupervised morphological segmentation algorithm outperforms BPEs consistently and that, although supervised methods achieve better segmentation scores, they under-perform in MT challenges. Finally, we contribute two new morphological segmentation datasets for Raramuri and Shipibo-Konibo, and a parallel corpus for Raramuri--Spanish.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSegmentationTranslationSimilar Papers 제목 키워드 기반
The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation
This paper evaluates the performance of several modern subword segmentation methods in a low-resource neural machine translation setting. We compare segmentations produced by applying BPE at the token or sentence level w…
Low Resource Neural Machine TranslationLow-Resource Neural Machine TranslationMachine TranslationSegmentation+2Quantifying Synthesis and Fusion and their Impact on Machine Translation
Theoretical work in morphological typology offers the possibility of measuring morphological diversity on a continuous scale. However, literature in Natural Language Processing (NLP) typically labels a whole language wit…
DiversityMachine TranslationTranslationQuantifying Synthesis and Fusion and their Impact on Machine Translation
Theoretical work in morphological typology offers the possibility of measuring morphological diversity on a continuous scale. However, literature in NLP typically labels a whole language with a strict type of morphology,…
DiversityMachine TranslationTranslationExploring Segmentation Approaches for Neural Machine Translation of Code-Switched Egyptian Arabic-English Text
Data sparsity is one of the main challenges posed by code-switching (CS), which is further exacerbated in the case of morphologically rich languages. For the task of machine translation (MT), morphological segmentation h…
Machine TranslationSegmentationImproving the Performance of Neural Machine Translation Involving Morphologically Rich Languages
The advent of the attention mechanism in neural machine translation models has improved the performance of machine translation systems by enabling selective lookup into the source sentence. In this paper, the efficiencie…
DecoderMachine TranslationSentenceTranslation