Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translation
The language-independency of encoded representations within multilingual neural machine translation (MNMT) models is crucial for their generalization ability on zero-shot translation. Neural interlingua representations have been shown as an effective method for achieving this. However, fixed-length neural interlingua representations introduced in previous work can limit its flexibility and representation ability. In this study, we introduce a novel method to enhance neural interlingua representations by making their length variable, thereby overcoming the constraint of fixed-length neural interlingua representations. Our empirical results on zero-shot translation on OPUS, IWSLT, and Europarl datasets demonstrate stable model convergence and superior zero-shot translation results compared to fixed-length neural interlingua representations. However, our analysis reveals the suboptimal efficacy of our approach in translating from certain source languages, wherein we pinpoint the defective model component in our proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationTranslationSimilar Papers 제목 키워드 기반
TARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognition
In this paper we propose a novel Temporal Attentive Relation Network (TARN) for the problems of few-shot and zero-shot action recognition. At the heart of our network is a meta-learning approach that learns to compare re…
Action RecognitionFew-Shot action recognitionFew Shot Action RecognitionMeta-Learning+4A neural interlingua for multilingual machine translation
We incorporate an explicit neural interlingua into a multilingual encoder-decoder neural machine translation (NMT) architecture. We demonstrate that our model learns a language-independent representation by performing di…
DecoderMachine TranslationNMTSentence+2High-Dimensional Interlingual Representations of Large Language Models
Large language models (LLMs) trained on massive multilingual datasets hint at the formation of interlingual constructs--a shared subspace in the representation space. However, evidence regarding this phenomenon is mixed,…
Lump at SemEval-2017 Task 1: Towards an Interlingua Semantic Similarity
This is the Lump team participation at SemEval 2017 Task 1 on Semantic Textual Similarity. Our supervised model relies on features which are multilingual or interlingual in nature. We include lexical similarities, cross-…
Language IdentificationMachine TranslationSemantic SimilaritySemantic Textual Similarity+1Multilingual and Interlingual Semantic Representations for Natural Language Processing: A Brief Introduction
We introduce the Computational Linguistics special issue on Multilingual and Interlingual Semantic Representations for Natural Language Processing. We situate the special issue{'}s five articles in the context of our fas…
Articles