Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders
In this paper, we propose a multilingual unsupervised NMT scheme which jointly trains multiple languages with a shared encoder and multiple decoders. Our approach is based on denoising autoencoding of each language and back-translating between English and multiple non-English languages. This results in a universal encoder which can encode any language participating in training into an inter-lingual representation, and language-specific decoders. Our experiments using only monolingual corpora show that multilingual unsupervised model performs better than the separately trained bilingual models achieving improvement of up to 1.48 BLEU points on WMT test sets. We also observe that even if we do not train the network for all possible translation directions, the network is still able to translate in a many-to-many fashion leveraging encoder{'}s ability to generate interlingual representation.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingNMTTranslationSimilar Papers 제목 키워드 기반
NIT Rourkela Machine Translation(MT) System Submission to WAT 2022 for MultiIndicMT: An Indic Language Multilingual Shared Task
Multilingual Neural Machine Translation (MNMT) exhibits incredible performance with the development of a single translation model for many languages. Previous studies on multilingual translation reveal that multilingual …
DecoderMachine TranslationNMTTranslationMultilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders
State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. In this paper, we propose an alternative approach that is based o…
DecoderLifelong learningMachine TranslationTranslationEvaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval
Pretrained multilingual text encoders based on neural Transformer architectures, such as multilingual BERT (mBERT) and XLM, have achieved strong performance on a myriad of language understanding tasks. Consequently, they…
Cross-Lingual Word EmbeddingsRepresentation LearningRetrievalSentence+2Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation
Acoustic word embeddings (AWEs) are fixed-dimensional representations of variable-length speech segments. For zero-resource languages where labelled data is not available, one AWE approach is to use unsupervised autoenco…
Contrastive LearningWord EmbeddingsMultilingual Neural Machine Translation With the Right Amount of Sharing
Large multilingual Transformer-based machine translation models have had a pivotal role in making translation systems available for hundreds of languages with good zero-shot translation performance. One such example is t…
DecoderMachine TranslationNMTTranslation