Neural Machine Translation of Rare Words with Subword Units
Neural machine translation (NMT) models typically operate with a fixed vocabulary, but translation is an open-vocabulary problem. Previous work addresses the translation of out-of-vocabulary words by backing off to a dictionary. In this paper, we introduce a simpler and more effective approach, making the NMT model capable of open-vocabulary translation by encoding rare and unknown words as sequences of subword units. This is based on the intuition that various word classes are translatable via smaller units than words, for instance names (via character copying or transliteration), compounds (via compositional translation), and cognates and loanwords (via phonological and morphological transformations). We discuss the suitability of different word segmentation techniques, including simple character n-gram models and a segmentation based on the byte pair encoding compression algorithm, and empirically show that subword models improve over a back-off dictionary baseline for the WMT 15 translation tasks English-German and English-Russian by 1.1 and 1.3 BLEU, respectively.
Code (26)
Tasks
Machine TranslationNMTSegmentationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Optimizing Segmentation Granularity for Neural Machine Translation
In neural machine translation (NMT), it is has become standard to translate using subword units to allow for an open vocabulary and improve accuracy on infrequent words. Byte-pair encoding (BPE) and its variants are the …
Machine TranslationNMTTranslationFinding Better Subword Segmentation for Neural Machine Translation
For different language pairs, word-level neural machine translation (NMT) models with a fixed-size vocabulary suffer from the same problem of representing out-of-vocabulary (OOV) words. The common practice usually replac…
Machine TranslationNMTSegmentationTranslationModeling Multi-granularity Segmentation for Rare Words in Neural Machine Translation
Segmenting rare words into subwords has become a commonly used and effective way to alleviate the open vocabulary problem in Neural Machine Translation (NMT). The existing dominant segmentation methods either give rare w…
DiversityMachine TranslationNMTSegmentation+1Multitask Learning For Different Subword Segmentations In Neural Machine Translation
In Neural Machine Translation (NMT) the usage of subwords and characters as source and target units offers a simple and flexible solution for translation of rare and unseen words. However, selecting the optimal subword s…
DecoderMachine TranslationNMTSegmentation+1An Analysis of BPE Vocabulary Trimming in Neural Machine Translation
We explore threshold vocabulary trimming in Byte-Pair Encoding subword tokenization, a postprocessing step that replaces rare subwords with their component subwords. The technique is available in popular tokenization lib…
Machine TranslationTranslation