Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages
Multilingual neural machine translation systems learn to map sentences of different languages into a common representation space. Intuitively, with a growing number of seen languages the encoder sentence representation grows more flexible and easily adaptable to new languages. In this work, we test this hypothesis by zero-shot translating from unseen languages. To deal with unknown vocabularies from unknown languages we propose a setup where we decouple learning of vocabulary and syntax, i.e. for each language we learn word representations in a separate step (using cross-lingual word embeddings), and then train to translate while keeping those word representations frozen. We demonstrate that this setup enables zero-shot translation from entirely unseen languages. Zero-shot translating with a model trained on Germanic and Romance languages we achieve scores of 42.6 BLEU for Portuguese-English and 20.7 BLEU for Russian-English on TED domain. We explore how this zero-shot translation capability develops with varying number of languages seen by the encoder. Lastly, we explore the effectiveness of our decoupled learning strategy for unsupervised machine translation. By exploiting our model's zero-shot translation capability for iterative back-translation we attain near parity with a supervised setting.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual Word EmbeddingsMachine TranslationSentenceTranslationUnsupervised Machine TranslationWord EmbeddingsSimilar Papers 제목 키워드 기반
Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation
End-to-end Speech Translation (ST) aims at translating the source language speech into target language text without generating the intermediate transcriptions. However, the training of end-to-end methods relies on parall…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)cross-modal alignmentMachine Translation+4Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performance
Multilingual Neural Machine Translation (MNMT) facilitates knowledge sharing but often suffers from poor zero-shot (ZS) translation qualities. While prior work has explored the causes of overall low ZS performance, our w…
Machine TranslationNMTTranslationImproving Zero-Shot Translation of Low-Resource Languages
Recent work on multilingual neural machine translation reported competitive performance with respect to bilingual models and surprisingly good performance even on (zeroshot) translation directions not observed at trainin…
Machine TranslationTranslationDeCo-DETR: Decoupled Cognition DETR for efficient Open-Vocabulary Object Detection
Open-vocabulary object detection (OVOD) enables models to recognize objects beyond predefined categories, but existing approaches remain limited in practical deployment. On the one hand, multimodal designs often incur su…
Object DetectionExploring Linguistic Similarity and Zero-Shot Learning for Multilingual Translation of Dravidian Languages
Current research in zero-shot translation is plagued by several issues such as high compute requirements, increased training time and off target translations. Proposed remedies often come at the cost of additional data o…
DecoderMachine TranslationTranslationTransliteration+1